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Introduction 



R ead Perspectives presents two major areas of focus in the current 
issue: varieties of English Acquisition programs and their costs, 
as published in a study prepared for the Arizona Department of 
Education, and a trio of reports on accountability for the aca- 
demic progress of English Language Learners (ELLs), formerly designat- 
ed Limited- English Proficient (LEP) students. Volume VIII concludes 
with a review of a survey on the monitoring of school districts with bilin- 
gual programs by the Office for Civil Rights, U. S. Department of Educa- 
tion. 

Arizona Department of Education 
English Acquisition Program Cost Study 

In February 2001, the Arizona Department of Education awarded a joint 
contract to the READ Institute and Sjoberg Eveshenk Consulting, LLC, 
to complete a study in four parts by May. Although the time allotted was 
very short, the full report covering the following areas was presented to the 
Department on time: 

Phase I — ^Program descriptions of six English Immersion models identified 
as successful in educating English Language Learners in U.S. schools and 
an analysis of the costs of each. 

Phase II — A comprehensive cost study of all resources spent and funds des- 
ignated for providing educational services to LEP students in Arizona 
schools, during the 1999-2000 school year, and a projection of the amount 
of funding that could be available for English Acquisition programs in fis- 
cal year 2002. 

Phase III — The costs and description of a model bilingual program in 
Arizona, in the Creighton elementary district. 

Phase IV — ^A detailed study of the program elements and cost components 
that address the unique needs of English Language Learners in the 
Nogales, Arizona, Unified School District. 

Phases I and IV are reprinted in full in this volume; the other segments 
are available from the CEO/READ web site (www.ceousa.org). 

The six districts identified in Phase I as having successful programs 
include the Bethlehem Area School District (Pennsylvania), Glendale 
High School and Thunderbird High School, Glendale (Arizona), Houston 
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Independent School District RITE Program (Texas), Phoenix Advantage 
Charter School (Arizona), and Seattle Public Schools’ Bilingual 
Orientation Program (Washington). The terms “English Immersion” and 
“English Acquisition” are used interchangeably in this publication. By 
either title, it is a program in which children are taught the English lan- 
guage from the first day of school, as well as being taught school subjects in 
English through a modified curriculum, with the expectation that students 
will master second language skills and make a rapid transition to main- 
stream classrooms. 

Cost studies of programs for bilingual students are not only rare but 
mostly inaccurate. The crucial lessons to be learned from this unique study 
of English-teaching programs and their costs may be summarized as fol- 
lows: 

■ The study dispels the notion that there are no English Immersion pro- 
grams in operation and that hardly anyone knows how to implement such 
a model. 

■ It provides models for distinctly different approaches useful in large or 
small districts, with children from one or many different native language 
backgrounds, and for children arriving at different grade levels from pri- 
mary school to senior high school. 

■ The range of costs varies from no special cost for the Advantage Charter 
School model to a high of S3, 000 per student per year for the Seattle Or- 
ientation School. 

Phase IV, the study of the Nogales Unified School District, presents an 
unusual case — a school district with 6,500 students, 80 percent of whom 
start school with a limited knowledge of English and 78 percent of whom 
are from families of poverty. Nogales sits on the U.S.-Mexican border, yet 
90 percent of its students are native-born US. citizens. In this mainly 
Spanish-speaking community, researchers found a strong preference for 
English Immersion teaching, the exact opposite of researchers’ expecta- 
tions. 

Although the report did not set out to compare achievement levels across 
different programs, this is the main finding that emerged firom the study: 
Elementary schools with English Immersion teaching produced higher stu- 
dent test scores and tested a much higher percentage of their LEP students 
than schools using bilingual education methods. In fact, in the schools with 
English Immersion programs, 100 percent of the students took the state- 
wide tests each year. The longer the English teaching program was in place, 
the higher the achievement scores of students on the reading, language, and 
math tests in English, a finding that is clearly documented in the individ- 
ual school profiles. 
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San Francisco Unified School District et aL v. 

State Board of Education et aL 

California, the state with almost half (43 percent) of the students classified 
as English Language Learners in the country, is also the place where four 
school districts resisted the state law requiring testing of all students annu- 
ally. The four districts came to settle their dispute with the state just days 
before a full court trial was due to begin on November 6, 2000. Perhaps the 
arguments on the side of the benefits of testing academic progress of 
English Language Learners outweighed the resistance to accountability for 
bilingual students and may have swayed the districts to settle. 

In April 1998, the California Department of Education, Superintendent 
of Public Instruction, and State Board of Education filed suit against the 
San Francisco Unified School District since the district refused to admin- 
ister the statewide tests to English Language Learners who had been in its 
schools less than 30 months unless a parent or teacher requested otherwise. 
State law required all students in grades 2-1 1 to participate in the annual 
exams, unless excused by written request of their parents. By June 1998, San 
Francisco Unified School District was joined by Oakland, Berkeley, and 
Hayward Unified School Districts in a countersuit against the state educa- 
tion agencies. 

Complainants and Defendants prepared for a court trial, with experts on 
both sides presenting arguments. As an expert witness on behalf of the 
California State Board of Education in this case, and as a long-time advo- 
cate of greater accountability for the academic progress of English 
Language Learners, I confess my partiality for universal application of tests. 
The crux of the matter is whether the State of California could require all 
students — including those who may have entered school without a suffi- 
cient knowledge of English to do regular class work in English — to partic- 
ipate in the new accountability program. Standardized Testing and 
Reporting (STAR). In the words of the STAR program guidelines, “It is the 
intent of the Legislature in enacting this chapter to provide a system of 
individual assessment of pupils that has, as its primary purpose, assisting 
pupils, their parents, and teachers to identify individual academic strengths 
and weaknesses, in order to improve teaching and learning.” (Phillips, 3) 
[California Education Code Section 60602(a)] 

California makes allowances for students new to the state, or to a partic- 
ular school district, whether they are limited-English or not. Any student 
who was not enrolled in a school district before June 30 of the previous year 
must take the state tests, but the test scores are not counted except as base- 
line information. After a full year in a district, the state accountability sys- 
tem begins to chart individual student progress and rates of improvement 
by schools and districts. There are new funds to help schools that are 
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deemed to be “under-performing” and rewards for individual teachers and 
districts that exceed expectations,’ but these extra infusions of state help 
cannot be apportioned without test scores that reflect the performance of 
all the students. Without uniform measurements, bilingual childrens 
achievement would not be charted and their needs would be neglected. 

San Francisco and the other three districts reached a settlement with the 
State of California that henceforth all English Language Learner students 
enrolled in California public schools in grades 2-11 shall take the achieve- 
ment tests under the STAR system. The settlement of this case in favor of 
the state’s right to assess the learning of all students is a major advance in 
the effort to include bilingual children fuUy in the education system, with 
all its rights, privileges, and responsibilities. Open, public reporting of stu- 
dent achievement, year after year, leads to making schools, administrators, 
teachers, and parents accountable for students’ learning and identifies areas 
where improvements are necessary. 

Of all the expert witness declarations, we have chosen to reprint the 
report prepared by Dr. Susan E. Phillips, an independent psychometric and 
assessment law consultant and former Michigan State University professor. 
Her report on behalf of the California State Board of Education provides a 
clear analysis of all the arguments and counterarguments, leading to her 
professional opinion that English Language Learners benefit from inclu- 
sion in the STAR program. Dr. Phillips’s 11 tables and 92 charts accompa- 
nying her report are available from the READ office on request. A copy of 
the final settlement in the case is also reprinted. 

All That Glitters Is Not Gold; 

The Limits of the Department of Education’s 
English Learner Achievement Data 

Professor Christine Rossell contributes this review of a research study pub- 
lished in December 2000 alleging that in at least the 63 bilingual schools 
sampled, the achievement of Hispanic students “appears to be at least as 
strong, if not better, than in schools providing a program overwhelmingly 
in English.... Bilingual education has not been a barrier to academic 
achievement in English, as measured by the SAT-9, and may have helped.” 
(Gold, p. 5) Such results, if correctly reported, would seem to contradict the 
published reports of the last three years that districts providing English 
Immersion teaching actually have shown greater gains in test scores for 
LEP students than those districts continuing their Spanish-language bilin- 
gual programs. 

Rossell draws a painstaking portrait of the defects in the data collection 
by the California Department of Education that impose real limitations on 
the ability of researchers to draw conclusions. The statewide testing pro- 
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gram in California has many admirable features, but the gaps in data col- 
lection for English Language Learner students are of serious proportions. 
It is difficult, if not almost impossible, to compare the test scores of students 
in bilingual education to those in all-English instruction in the same school 
because the test scores are not broken down by program or ethnicity. The 
percentage of students tested varies greatly from school to school, while it 
is often the case that testing rates are much higher for all-English programs 
than for bilingual programs. There are many other problems that emerge in 
the analysis of the Gold study. Rossell concludes that the data themselves 
have serious limitations and one must be cautious in drawing conclusions 
from them.” (Rossell, 14) 

Seven Successful Bilingual Schools in Texas 

A new contributor to READ Perspectives^ Dr. John Correiro of Brown Uni- 
versity, reviews the study released by the Texas Education Agency in August 
2000, Texas Successful Schools Study: Quality Education for Limited English 
Proficient Students. A collaborative project of the Texas Education Agency, 
the Charles A. Dana Center at the University of Texas at Austin, and Texas 
A 8cM University — Corpus Christi, the study identifies seven' elementary 
schools reporting higher than average academic performance. Each school 
enrolls a student body that has 40 percent or more LEP students, of whom 
50 percent or more are from economically disadvantaged families. The 
study covers the school years between 1994-95 and 1998-99. High testing 
rates are recorded, with the unusual achievement in 1996-97 that all stu- 
dents participated in the Texas Assessment of Academic Skills (TAAS). 

In recent years the successful schools literature has begun to take account 
of the achievements of schools enrolling large numbers of minority students 
from economically disadvantaged families, as in the READ Perspectives 
review of the “No Excuses Campaign” (Rossier). There are valuable, practi- 
cal lessons for school districts across the country in the descriptive materi- 
als provided in the Texas study. Based on extensive interviews with teach- 
ers, administrators, and parents, surveys, site visits to classrooms, and analy- 
ses of test scores, the study provides a detailed description of the programs, 
instructional practices, and school characteristics that resulted in the 
implementation of high standards and high expectations for these children. 
The complete study is available on request from the READ Institute office. 

Critique of GAO Report: Meeting the Needs 
of Students With Limited English Proficiency 

Once again the U. S. General Accounting Office (GAO) has set itself the 
task of surveying the field of education for language minority children, 
principally English Language Learners in U.S. public Schools. The last 
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such study by the GAO, Limited English Proficiency: A Growing and Costly 
Educational Challenge Facing Many School Districts, appeared in January 
1994 and was reviewed by me in the Spring 1995 issue of READ 
Perspectives. It was my opinion at that time that the GAO did not fully 
answer the questions set out in the “challenge.” The current review by Jim 
Littlejohn of the new study again finds the GAO falling short of its goal. 
Three questions are investigated but not satisfactorily answered: 

1. How long do LEP children need to become proficient in English? 

2. What instructional approaches are used and how long do LEP students 
remain in language assistance programs? 

3. What requirements does the Office for Civil Rights (OCR) expect 
school districts to meet, how are they set forth, and what is the nature of 
the interactions between OCR and the districts? 

When it was first published in February 2001, the report appeared to 
clear the OCR of any charges of heavy-handedness in pressuring school 
districts to implement native language instruction programs, a complaint 
heard among educators for a dozen years. Education Week immediately pro- 
claimed the exoneration, “OCR Seen as Unbiased on Bilingual Ed. Issue.” 
(Wright, 3) Littlejohn, himself a former OCR administrator, cuts through 
the bureaucratic language of the report to provide a cogent analysis of its 
serious shortcomings, especially in the whitewash of OCR. We welcome 
Mr. Littlejohn to the pages of READ Perspectives. 

— Rosalie Pedalino Porter, EdD., Editor 
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The Cost 

of English Acquisition 
Programs 

Arizona Department of Education 
English Acquisition Cost Study 

Phase I 

The Costs 

OF English Immersion 
IN Six Model Programs 

Executive Summary 

I n February 2001, the Arizona Department of Education (ADE) con- 
tracted with two groups for a cost study of six English Immersion pro- 
grams. During the five weeks of this phase of the study, one contrac- 
tor, the Research in English Acquisition and Development Institute 
(READ Institute), was tasked to provide qualitative program analysis for 
six English Immersion programs. The ADE selected three Arizona public 
schools for review: Phoenix Advantage Charter School, Thunderbird High 
School, and Glendale High School. Additionally, the ADE and READ 
agreed upon three other out-of-state programs: Bethlehem Area School 
District, Bethlehem, Pennsylvania; the Rodeo Institute for Teacher Excel- 
lence (RITE) Program, Houston, Texas; and the Seattle Public Schools’ Bi- 
lingual Orientation Centers, Seattle, Washington. The ADE simultaneous- 
ly contracted with a second firm, Sjoberg Evashenk Consulting, LLC, to 
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identify and analyze the cost elements related to these six programs and to 
project the resulting costs into 2002 Arizona cost of living dollars. 

This report conveys the results of Phase I of the cost study virhere the two 
groups identified the key program and cost components of the selected 
model English Immersion programs. Generally, the six immersion pro- 
grams employ some different philosophies and techniques, yet share many 
similarities on the road toward fulfilling comparable goals. For instance, 
although each of the six programs are classified as English Immersion, not 
all exercise the same instructional methods; some embrace Direct 
Instruction — some for the full curriculum while others apply that technique 
for only language arts; and still others follow a structured immersion pro- 
gram whereby language arts focuses on English Acquisition and students 
are mainstreamed during the remainder of the day. Finally, many of the 
selected programs “shelter” extremely limited English proficient students, 
termed “beginners,” for a short time (typically six months to a year) to pro- 
vide extensive English instruction to assist students to quickly acquire a 
basic level of English so they may move into the mainstream student envi- 
ronment. 

The READ Institute’s review found common elements across aU pro- 
grams. Specifically, many programs use reduced size classes during special- 
ized English instructional time, apply phonics-based reading and Direct 
Instmction teaching techniques, and provide high levels of ongoing teacher 
training. In addition, many of these programs employ bilingual teachers or 
aides who may use the students’ native language on occasion to ease the 
transition process. Further, all instruction in reading, writing, and other 
school subjects is given in a special Content-Based English Teaching cur- 
riculum. In each program surveyed, teachers held high expectations for lim- 
ited English proficient (LEP) students and these students were taught the 
same curriculum as the mainstream students, albeit at a slightly modified 
pace. 

Sjoberg Evashenk Consulting, LLC, was tasked with identifying all the 
cost elements related to delivering the selected model English Immersion 
programs. Overall, the districts and charter schools report that most of the 
costs related to providing teachers and other services necessary for serving 
LEP students are not unlike those incurred in delivering services for “main- 
stream” students. Moreover, Sjoberg Evashenk found that several of the 
programs approach teaching LEP students in similar ways: for example, the 
Bethlehem and Glendale districts target English acquisition services 
through their language arts courses and mainstream the LEP students for 
remainder of the day. These English Acquisition language arts programs are 
taught in lieu of “mainstream” courses and electives and typically consume 
most of the students’ day. Notwithstanding the replacement of other 
course- work, the additional services and assistance that accompany these 
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districts’ programs generate some identifiable incremental costs. In another 
instance, the Seattle Bilingual Orientation Centers (BOCs) and Phoenix 
Advantage provide “beginners” classes that for a time “shelter” LEP stu- 
dents from the mainstream — again, the majority of the classroom costs of 
these offerings replace the costs that the district would normally incur to 
educate these pupils. 

Of the selected programs reviewed, one element of incremental cost 
incurred by these programs relates to reduced class sizes for LEP students. 
At schools in the Glendale district, class sizes for “beginner” LEP students 
average about 31 percent smaller than the average mainstream classes. In 
contrast, Bethlehem approaches class size from another perspective. This 
district reduces the size of all classes in a school by nearly 17 percent when 
it is determined to be “high or medium impact” — meaning the number of 
LEP students in the school exceeds approximately 12 percent of total stu- 
dent enrollment. However, Sjoberg Evashenk found that the Seattle BOCs, 
Phoenix Advantage, and the Houston RITE program do not reduce class 
sizes even in those groups considered beginners. 

Additionally, English Acquisition programs commonly employ class- 
room aides to assist LEP students; these costs are easiest to identify.. While 
several of the programs devote at least one aide for each English 
Acquisition classroom, it is important to note that some districts and 
schools provide classroom aides (available to assist all children) as part of 
their core method of providing regular educational services. For example, 
Phoenix Advantage’s Direct Instruction program provides an aide in every 
classroom as an essential element for delivering the educational program. 
Further, the RITE program is premised upon having “trainers” available 
daily to provide assistance to classroom teachers and the students partici- 
pating in their reading program. 

Overall, Sjoberg Evashenk found the costs of providing these English 
acquisition programs vary vvddely. Specifically, Glendale and Bethlehem 
both appear to have comprehensive, schoolwide English Immersion pro- 
grams currently reporting incremental costs of $983 and $1,056 per LEP 
student, respectively. Although the Phoenix Advantage and Houston RITE 
programs both employ Direct Instruction programs, only Phoenix 
Advantage applies this approach to its full-scope educational program. 
Sjoberg Evashenk’s review reveals that Phoenix Advantage incurs two 
modest incremental costs equating to nearly $185 per LEP student for the 
1999-2000 school year. RITE’s program, primarily a reading and language 
program targeted at pre- kindergarten though grade 2 children, costs its pri- 
vate, nonprofit organization sponsor approximately $1.3 million per year or 
approximately $238 per disadvantaged student served. Finally, the Seattle 
Public Schools’ Bilingual Orientation Centers that employ sheltered im- 
mersion techniques incur costs of more than $3,000 per LEP student. 
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Study Results 
at a Glance 




Selected Model English Acquisition Programs 
Identified Incremental Costs 

(Projected into 2002 Arizona cost-of-living dollars) 




■ Phoenix Advantage Charter School 

Uses Direct Instruction as its immersion model. 
Reports 209 or 24% enrollment as LEP. 

Identifies LEP costs related to assistants and testing. 
Projected 2002 incremental cost per LEP student 


$192.10 


■ Glendale Union High School District 

Applies English Immersion in language arts courses. 
Reports 648 or 5% participate in the LEP program. 
Incremental costs identified in 8 program elements. 
Projected 2002 incremental cost per LEP student 


$1,023.02 


■ Bethlehem Area School District 

English Immersion throughout district K-5. 
Reports 760 or 10,4% LEP student enrollment. 
Incurs incremental LEP costs in 7 areas. 
Projected 2002 incremental cost per LEP student 


$1,076.79 


■ Houston RITE Program 

Privately funded reading and language program. 
Targets under-performing Pfe-K through 2nd graders. 
In 20 schools serving 5,300 children. 

Projected 2002 RITE program costs per child 


$243.05 


■ Seattle Public Schools Bilingual Orientation Centers 

Sheltered Immersion program for LEP “beginners.” 

5 bilingual centers 6c 1 family center serve 450 students. 

Reports incremental K-12 LEP costs in 7 areas. 

Projected 2002 incremental cost per LEP student $3,067.91 
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Chapter! 

English Acquisition Model 
Program Review 



Chapter Summary 

I n Phase I of this program cost study the Arizona Department of 
Education asked The Institute for Research in English Acquisition 
and Development (READ) to focus on six specific programs for LEP 
students that can best be described as English immersion programs 
and to identify key elements of their success. Every one of these programs 
uses an English-teaching approach that does not rely on the students! native 
languages for classroom instruction, although few of these programs have 
all the elements of a classic structured/sheltered English immersion envi- 
sioned by some educators.^ In most cases, Limited English Proficient 
(LEP) students attend mainstream classrooms with native-English-speak- 
ers as much as possible, according to each student's level of English profi- 
ciency. This kind of flexibility allows students to move ahead or cover learn- 
ing materials as many times as necessary, according to each one’s cognitive 
ability and individual circumstances. 

The common elements across all programs are smaller class sizes during 
specialized English instmctional time, heavy use of phonics based reading 
programs, direct instruction, and high levels of ongoing teacher training. 
Many of these programs employ bilingual teachers or aides, who may use 
the students’ native language on occasion to ease the transition process. All 

*A Stxuctured/Sheltered English Immersion Program is based on the premise that children 
can learn a second language rapidly and effectively, and can learn school subjects taught in the 
second language almost from the first day of school. The essential features are: 

■ A self-contained classroom with all LEP students. 

■ Professional staff trained in structured immersion approaches. 

■ The teacher or aide may know the primary language of the students but does not use it for 
instruction purposes and always answers student questions in English. 

■ A special curriculum is employed which uses content-based language teaching strategies 
similar to ESL. Content and English are taught concurrendy. 

All instruction, print materials, texts, and videos are in English. 
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instruction in reading, writing, and other school subjects is given using con- 
tent-based teaching techniques which use a modified curriculum and visu- 
als to assist LEP students in learning English and understanding the cur- 
riculum simultaneously. In each program surveyed, teachers hold high 
expectations for LEP students, and students are taught the same curriculum 
as the mainstream students, albeit at a slightly modified pace. 

While some schools do provide pay incentives for English as a Second 
Language (ESL) certified teachers, most do not. Funds are often used on 
professional development for all teachers and teachers’ aides to teach them 
techniques to work with LEP students. A common characteristic of these 
programs is a school culture that views education for LEP students as being 
the responsibility of every teacher and teacher’s aide. Except for the Bethle- 
hem, Pennsylvania, English Acquisition Program, all the other schools 
studied have problems with shortages of teachers and aides, but few have 
problems retaining qualified teachers. 

We established that our criteria for program success would be reclassifi- 
cation rates and standardized assessment scores. The programs selected by 
the Arizona Department of Education and the READ Institute were cho- 
sen on the presumption that they were successful. Reclassification rates, or 
changing a student’s classification from LEP to non-LEP, is only one cri- 
terion for program success and may not always be the most accurate one, 
since students are often successfully exited from special programs to main- 
stream classrooms before they are technically reclassified. Research on pro- 
grams for LEP students shows that most LEP students are capable of per- 
forming mainstream classroom work well before they meet the arbitrary 
criteria for reclassification (Rossell, 2000). 

Reliance on standardized assessment tests to establish program effective- 
ness also has drawbacks. Most often, LEP students are exempted from 
standardized tests in EngHsh for several years and only those LEP students 
who are expected to pass the test take it. In the case of Pennsylvania, there 
is no requirement for students to take standardized tests, so test score data 
are not available. All districts with LEP students measure their students’ 
progress in learning English by using such tests as the Language 
Assessment Scales (LAS) to measure from Beginner to Intermediate to 
Advanced levels, along with teacher evaluations of student work, portfolios, 
and other measures. 

The common denominator among all of these programs is the conviction 
that teaching children English as quickly as possible allows them to suc- 
cessfully perform grade appropriate academic work in a classroom with 
their native-English-speaking peers. While LEP students in high school, 
especially those with no prior education, take longer to learn English, ele- 
mentary level students are mainstreamed within two years in most cases. 
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Phoenix Advantage Charter School 
LEP Program Grades K-8 

Johanna Haver 

Program Elements 

Phoenix Advantage Charter School is part of a national management com- 
pany that oversees fifteen schools in eight states. All Advantage schools 
share the same instructional model and curriculum that is based on Direct 
Instruction, also known as Distar. The materials include Reading Mastery, 
a systematic phonics program, Language for Learning, Spelling Mastery, 
Distar Arithmetic, and Connecting Math Concepts, all published by SRA. 
The lessons are scripted for the teachers in reading, language, and math. 
Students learn in small instructional groups based on academic skill level 
rather than grade. Their progress is evaluated almost on a weekly basis. 
Students are moved from group to group according to what skills they have 
mastered. Also, students participate in specially designed character, educa- 
tion programs and adhere to the standards of the President’s Council on 
Physical Fitness. Students follow a uniform dress code. 

Phoenix Advantage began as a free and public Arizona charter school in 
September of 1997 with 250 students in grades K-5. Now, in its fourth 
year, the school’s enrollment has reached 1,030 students and extends 
through grade 8. The ethnic make-up is 70 percent Hispanic, 15 percent 
Anglo, 13 percent African-American, and 2 percent other. There is one 
school director, six other professional staff, 40 teachers, and 19 instruction- 
al assistants. Many of the students come from families that are very mobile; 
thus the student body in the fall looks quite different from that in the 
spring. The school year lasts 200 days, which is 25 school days longer than 
the school year of traditional Arizona public schools. Also, the school day 
runs from 8:00 a.m. to 3:30 p.m., which is from 1 to 2 hours longer than 
that of most Arizona public schools. Phoenix Advantage received 
Presidential Academic Awards in both 1998 and 1999. Considering that 
most of these students are considered “at-risk,” the test results have been 
very good. 

As is the case with charter schools in general, enrollment is strictly vol- 
untary, and the instructional program is the same for all students whether 
LEP or not. The school uses the same structured and sequential approach 
for building language skills for all students, and all students follow the same 
scripted lessons. 

Only those LEP students with the lowest level of English in grades 2 to 
8 receive ESL instruction separate from mainstream classes. In grades K to 
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1, the regular program is designed in such a way that it addresses the needs 
of the very young LEP and non-LEP students in the same classroom. 

There are two non-graded ESL classes for the 60 beginning LEP stu- 
dents in grades 2 to 8. One contains 30 students in grades 2, 3, and 4; the 
other, 30 students in grades 5, 6, 7, and 8. A full-time instructional assis- 
tant is also assigned to each class. The curriculum for beginning ESL stu- 
dents is the same as for mainstream students, but presented in a manner 
especially designed for LEP students. Sometimes the LEP students in the 
beginning classes use the Horizon reading program, which hke Reading 
Mastery has a Direct Instruction format. 

LEP students receive instructional assistance in these beginning classes 
for most of the day. With few exceptions, LEP students exit beginning ESL 
in one year, although some students may continue in the program for an 
additional year. After completion of beginning ESL, these students are 
placed in mainstream classes. 

The format for all classes is that the teacher and/or the instructional 
assistant work direcdy with a group of 8 to 12 students while the rest of the 
class works independently at their desks. The students repeat in unison 
after the teacher in accordance with the scripted lesson. At the end of a 
task, the teacher gives students individual turns to check for student mas- 
tery of the skill that has been taught or introduced. The individual turns 
allow the teacher to know which students may need additional practice at 
the end of the lesson. 

Mastery tests are given approximately every 5 to 10 lessons, depending 
on the program. Mastery is achieved when 90 percent of the students score 
90 percent or better on the test. Students who do not receive 90 percent 
receive remedial assistance and are re-tested. Occasionally a student has to 
be moved to a lower group that moves more slowly, or to a higher group 
that moves more rapidly. Because each subject is scheduled at the same time 
for all classes, it is not difficult to make these adjustments. Because the 
school year and school day are unusually long, all remedial student needs are 
met within the regular school day/year. 

The school’s philosophy is based on the theory that children need a 
strong educational foundation and will succeed if presented with struc- 
tured, sequential lessons. Although many teachers and other staff members 
speak Spanish, the curriculum is in EngUsh except for a small amount of 
Spanish taught as a foreign language. The teachers understand that the stu- 
dents’ native language is not a barrier to their achievement. It is not accept- 
able to isolate LEP students or to expect less from them. 

LEP Statistics 

LEP students comprise 54 percent of the student body, with more than 500 
students designated as LEP. Many of the students have been unsuccessful 
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in other schools. It is estimated that 99 percent of the LEP students speak 
Spanish and 78 percent of the all students participate in the free or reduced 
price school lunch program. 

Staffing Model 

The student to teacher ratios are as follows: 

■ Kindergarten has 25 students to one teacher and one instructional assis- 
tant. 

■ First grade has 30 students to one teacher and one instructional assistant. 

■ Grades 2 to 8 have 30 students to one teacher. 

There is an average of 22 students per teacher or teacher s aide, and there is 
one aide per grade level for grades 2 to 8. 

Teachers are required to have at least a bachelor’s degree, and ESL 
endorsement is not required. However, bilingual skills in Spanish and Eng- 
lish are highly prized and at least two teachers have ESL endorsements. 

All teachers and instructional assistants serve LEP students. The profes- 
sional development for both groups includes ten days training at the begin- 
ning and five days throughout the school year. Informally, there is on-going 
professional development, mentoring, and guidance among teachers. 

There is a shortage of teachers. The recruiting strategy includes adver- 
tisements in the newspaper and sending individuals from the corporate 
office to teacher job fairs and teacher conferences to find employees. 
However, most of the teachers come to the school after hearing about it by 
word of mouth. 

The pay for teachers is comparable to that of other public school teach- 
ers, but these teachers work harder because of longer hours and the extend- 
ed school year. The pay schedule is merit-based and exemplary teacher per- 
formance can result in a 9-10 percent pay increase. There is an extensive 
point and rating system, and teachers are judged for the progress they make 
with lessons, their work ethic, their team spirit, their contributions to col- 
leagues, and their ability to communicate with parents. 

Assessment Statistics 

LEP students take the oral portion of the Idea Proficiency Test (IPT) twice 
a year and the curriculum mastery skill tests almost weekly. The IPT is 
helpful in that it offers documentation as to the students’ overall progress 
toward English language proficiency. 

The reclassification criteria have not been firmly established. 
“Reclassification” would be meaningless as to placement, because students 
are assessed on skill mastery frequendy and almost all LEP students are 
mainstreamed after one year. 
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Table I-l. 

Phoenix Advantage Charter School 
AIMS Results for All Students School Year 1999-2000 



Number of 
Students Tested* 



Mean Score 
Math 



Mean Score 
Readin 



Mean Score 
Writin 



77-80 475 510 518 



78-79 464 494 470 



*The number of students tested varied because different portions of the test were administered 
on different days and not all smdents were present for all portions of the test. 

The State of Arizona requires that the Arizona Instruments to Measure 
Standards (AIMS) Test be administered annually in the spring to all stu- 
dents in grades 3, 5, 8, and high school. When a student scores at least 500 
points on any of the three sections of the test, that student has met the stan- 
dard in that particular subject area. 



Table 1-2. 

Phoenix Advantage Charter School 
Spanish Version of the AIMS Results 
School Year 1999-2000 



Number of 
Students Tested 



Mean Score 
Math 



Mean Score 
Readin 



Mean Score 
Writin 



3 n 432 486 445 

5 9 4^ 474 469 

Note: The Spanish version of the AIMS test may be taken only once in place of the English ver- 



Table 1-3. 

Phoenix Advantage Charter School 
Stanford 9 Percentile Rank Scores — All Students 
School Year 1999-2000 



Number of 
students tested 



Percentile 
Rank Math 



Percentile 
Rank Readin 



Percentile 
Rank Lan 



: 143-188 


21 


38 ; 


117-120 


32 


38 


1 105-111 


24 


28 ; 


: 89-91 


18 


23 


. 75-76 


25 


31 ; 


^ 56-57 


30 


37 




The Stanford 9 is a standardized, nationally norm-referenced test administered annual- 
ly to students in Grades 2-12. The national, averse percentile rank score for each 



grade and each subject is the 50th percentile. LET students who attended school in 
Arizona for fewer than three years were exempt from this test. 

Arizona’s Measure of Academic Progress (MAP) is an indicator of stu- 
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dent academic growth from one year to the next. The results displayed in 
Table 1-4 are based on the Stanford 9, given in 1999 and 2000. MAP 
includes only those students who were tested both years in consecutive 
grade levels at the same school or who started the school year in the same 
school in which they were tested in 2000. 

Table 1-4. 

Phoenix Advantage Charter School 
MAP Results — ^AU Students 
School Year 1999-2000 



Grade 


Percent Expected 
Gain — Math 


Percentile 
Rank Math (1999) 


Percent Expected 
Gain — Reading: 


Percentile 

Rank Reading (1999) 


2nd-3rd 


152 


27 


139 


38 


i&ESSi 


113 


22 


121 


31 


!B!W 


88 


37 


154 


41 




200 


16 


197 


26 




114 


35 


123 


41 



Average student growth over the course of the past academic year is com- 
pared to a national average in the table above. One hundred percent indi- 
cates that students in a particular grade level at the school have achieved an 
average amount of growth compared to a national sample. A percentage 
greater than 100 percent indicates student growth was greater than the 
national average. 



Glendale High School LEP Program 
English Acquisition Program 
Grades 9-12 

Johanna Haver 

Program Elements 

Glendale High School is one of nine comprehensive secondary high 
schools in the Glendale High School District. One principal and three 
assistant principals oversee sixty-five teachers, six other professional staff, 
and nineteen teacher aides. Thirty-three of those teachers have a Master^s 
Degree or higher, and thirty-two of them have taught for ten years or 
longer. 

The enrollment of Glendale High School is 1,410 students with approx- 
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imately 140 identified as Limited English Proficient (LEP). The school is 
considered a majority-minority school with 50 percent Hispanic, 40 per- 
cent Anglo, 8 percent African-American, and 2 percent other. The general 
school population is highly transient with as many as 600 in and out dur- 
ing one school year. Nevertheless, the Stanford 9 Percentile Rank Scores 
indicate a trend of 9-10 percentile gains in reading and language from the 
9th to the 11th grade with, math scores remaining at the national norm. The 
AIMS results show a momentum that makes it likely that a large majority 
of the 10th and 11th graders will pass the reading and writing sections of 
the AIMS test in time for graduation. The math section remains a chal- 
lenge for these students, as it is for high school students throughout 
Arizona. 

Students have many opportunities to prepare themselves for college 
and/or work beyond high school, and 13 percent to 14 percent of the stu- 
dents take AP classes that include English literature, French, Spanish, U.S. 
history, U.S. government, physics, and calculus. The district pays for the AP 
testing of all students. Selected students participate in Ace Plus, a partner- 
ship with Glendale Community College that allows students to earn up to 
24 college credits while still in high school. Cisco Networking Academy, a 
two-year program, prepares students for internationally recognized certifi- 
cation in computer networking. Over 80 percent of the 1999 graduating 
class went on to college, and it is believed that 88 percent of the 2000 grad- 
uating class enrolled in postsecondary schooling also and 46 percent of the 
2000 graduates received college scholarships that equaled $3,581,800. 

Remedial help is available to students. The school offers extensive aca- 
demic assistance to aU students before and after school through Title- 1. 
ESL resource classes are offered to LEP students where aides and some- 
times computers are available. A peer- tutoring program is also in place that 
makes it possible for students to be trained to assist their fellow-classmates 
and earn a high school credit as weU. Many former or advanced ESL stu- 
dents are especially eager to participate in this program so that they can 
lend a hand to those students who are just beginning to learn English. 

Glendale High School uses a structured English immersion model for 
LEP instruction. The philosophy is that LEP students need to be taught 
English intensively to such a degree that they can move as soon as possible 
into the mainstream to participate in all the academic opportunities of 
Glendale High School. It is the policy of Glendale High School to explain 
to the parents of students identified as LEP that participation in the ESL 
program is voluntary. 

There is no sheltering outside of reading and English classes. ESL stu- 
dents are encouraged strongly by ESL teachers, content-area teachers, and 
administrators to participate in the many school activities. 

The ideal case progression of an LEP student’s education occurs when 
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that student has literacy in his or her native language. A literate, well-edu- 
cated child who speaks no English initially can move quickly through the 
ESL program in two years, and, after that, possibly qualify for honors and 
Advanced Placement (AP) classes. The worse case progression occurs when 
an LEP student has received no formal education. It is very difficult for any 
high school to make up for the lost time, especially when the instruction is 
in a language unfamiliar to the child. Generally, the child needs a great deal 
of remedial help. 

English as a Second Language (ESL) classes are offered in grades 9-12 
to students according to their particular level of English language profi- 
ciency. 

Beginning Level — ^3 periods per day - 15 students to one ESL endorsed 
teacher (20 max) and two aides. 

ESL Speaking 
ESL Reading 
ESL Writing 

Intermediate Level — 2 periods per day - 25.8 students to one ESL 
endorsed teacher (same as regular class) and one aide. 

ESL English 1-2 
ESL Reading 1-2 

Advanced Level — 2 periods per day - 25.8 students to one ESL endorsed 
teacher (same as regular class) and one aide. 

ESL English 3-4 
ESL Reading 3-4 

ESL Resource — 1 period per day - 25.8 students to one ESL endorsed 
teacher (same as regular class) and one aide. 

Beginning ESL Speaking. The instruction concentrates on building ba- 
sic listening, speaking, reading, and writing skills. Students are taught how 
to pronounce words clearly, follow oral directions and commands, and to 
respond appropriately to oral questions. The students learn to use target 
vocabulary and simple grammatical forms correctly while speaking on a 
variety of topics. This includes spontaneous conversation, prepared dia- 
logues, and speeches. 

Beginning ESL Writing. Beginning students are taught simple and 
compound sentence structure, usage, and mechanics. The correct use of tar- 
get vocabulary in context is emphasized. Composition pieces introduced 
include a brief personal narrative and/or poem, short summary or factual 
information, personal letter, brief personal information card, simple appli- 
cation, and explanatory paragraph. 

Beginning ESL Reading. Students develop oral and silent reading 
skills. While reading fiction and non-fiction, students practice reciprocal 
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reading strategies, the process of reading paragraphs orally in pairs and then 
asking each other questions about those paragraphs. Extensive vocabulary, 
selected literary terms, and selected literary elements are introduced. 
Students are taught to analyze both fiction and non-fiction for overall 
meaning. 

During the rest of the day the beginning students are mainstreamed into 
classes that do not require print-based material. They include math, PE, 
art, and music. 

Intermediate Level ESL English 1-2* Students develop English Avriting 
proficiency along with effective oral presentation skills and reading strate- 
gies. The composition pieces include the explanatory essay, summary, per- 
sonal narrative, and personal letter. Students learn to apply correct conven- 
tions to their writing and to locate resources in the media center. The cur- 
riculum is based on the 9th grade language arts curriculum. 

Intermediate Level Reading 1-2. Development of reciprocal reading 
strategies for both fiction and non-fiction is encouraged. This is a process 
proven highly effective in which pairs of students read orally back and forth 
to each other, one paragraph at a time, and then ask questions of each other. 
The study of literature is introduced by reading short stories, poetry, nov- 
els, and/or plays. Students are taught to analyze the short story and poetry 
for literary elements, figurative language, and overall meaning. Oral and 
written communication skills are included. 

During the rest of the day, the intermediate students are mainstreamed 
into history and introductory science as well as math, PE, art, and/or music. 

Advanced Level ESL English 3-4. Similar to English 1-2, but present- 
ed at a higher level and with more content. The curriculum is based on the 
10th grade language arts curriculum. 

Advanced Level ESL Reading 3-4. Similar to Reading 1-2, but pre- 
sented at a higher level and with more content. 

During the rest of the day, the advanced students take such classes as 
U.S. history, biology, and computer classes. No class is closed to these stu- 
dents. Many take advanced-math classes, honors classes, and Advanced 
Placement (AP) calculus. 

ESL Resource. This class consists of an ESL teacher, an instructional 
aide, and peer tutors to provide academic support for LEP students in 
mainstream classes. Study sldUs and test taking strategies are taught, and 
access to computers is available at selected sites. 

AH LEP students are encouraged to take pre-algebra at summer school 
for six weeks. This is available to aU students. ESL classes are offered at 
summer school to LEP students who enrolled as beginners late in the 
school year and/or need to work on literacy. The ESL classes last for five to 
six weeks at one campus, but are open to LEP students from aU of the dis- 
trict schools. The ESL students are not charged; separate district funding is 
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available for this. 



LEP Statistics 

Approximately half of the entering LEP students in the entire district come 
from feeder schools and the other half come from other countries. The total 
number of LEP students equaled 103 for the 1999-2000 school year and 
140 for the 2000-2001 school year. 

Table 1-5. 

Glendale High School 

Distribution of Students According to Grade and ESL Class 
School Year 2000-2001 



Grade Level 


9th grade 






12th grade 


Classes 


Sections 


Beginning 
3 hrs per class 


50 


13 


1 


0 


4 


12 


Intermediate 
2 hrs per class 


18 


21 


4 


0 


2 


4 


Advanced 
2 hrs per class 


0 


4 


13 


1 


1 


2 .. 


Resource 1 hr 


6 


16 


9 




2 


2 


Total 


74 


54 


27 


1 


9 


20 



The total numbers on the above table exceed the total number of LEP students due to an over- 
lap with the resource groups. The classes of students are the groups that meet together for 1-3 
hours each day. Each of those hours counts as a separate section. Thus, nine classes, or groups, 
attend 20 sections. 



For the 1999-2000 school year, 90 LEP students spoke Spanish (approx- 
imately 87 percent). Two students spoke French, one student spoke 
Punjabi, one student spoke Khmer, three students spoke Arabic, two stu- 
dents spoke Serbo-Croatian/Bosnian, one student spoke Tagalog, and three 
students spoke “other.” 

For the 2000-2001 school year, 120 LEP students speak Spanish 
(approximately 86 percent). Five students speak French, one student speaks 
Punjabi, two students speak Vietnamese, one student speaks Arabic, seven 
students speak Serbo-Croatian/Bosnian, one student speaks Tagalog, one 
student speaks Persian, one student speaks Swahili, and one student speaks 
“other.” 

Language Line from Monterey provides 24-hour interpreters for teacher 
to use a 3 -way conversation in which the language of the parent or guardian 
is quickly translated. However, it costs S2.50 per minute, and teachers are 
asked to keep the conversations short. Family literacy classes are offered to 
parents at Glendale HS and funded through a Tide-1 family literacy pro- 
gram. 
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No students receive either reduced price or free school lunch. However, 
students may work in the cafeteria for pay and free meals. 

According to data collected from the students during the 1999-2000 
school year, the following work patterns exist among all students at 
Glendale HS: 8 percent have full-time jobs; 34 percent have part-time jobs; 
and 3 percent have both full-time and part-time jobs. 56 percent of the stu- 
dents do not have jobs. The school officials believe that the percentages of 
LEP students who work are considerably higher than those of the overall 
school population. 

Staffing Model 

The student to teacher ratio for LEP students at the beginning level is from 
15 to 20 students to one ESL teacher; the intermediate and advanced lev- 
els have the same ratios as exist in mainstream classes, 25.8 to one teacher. 
AH five teachers of ESL classes hold ESL endorsements. AH teachers serve 
LEP students. The content-area teachers backgrounds in ESL vary. How- 
ever, aU content-area teachers have access to ESL in-service. They are 
encouraged, but not required, to have ESL endorsements. 

Each school has a “mentor” teacher who helps aH 1st, 2nd, and 3rd year 
teachers — regardless of a new teacher’s former experience. That mentor 
teacher has three class periods free to instruct the teachers concerning class- 
room management, cooperative learning. Effective Elements of Effective 
Instruction (EEEI), and other teaching strategies that are helpful to aU 
teachers. This districtwide mentoring program has been recognized nation- 
aHy. 

Special ESL workshops are provided to aH teachers throughout the 
school year. This is arranged through Karen Henderson, the ESL coordi- 
nator (a 3/5 rather than fuU-time position). She is often asked to teach 
strategies designed especiaUy for LEP students to a team of content-area 
teachers. Furthermore, as teachers practice these strategies, they share what 
works with their coHeagues. Both non-ESL and ESL endorsed teachers re- 
ceive a lot of cross-development specialized staff development. The teach- 
ers who wish to attend summer staff development are paid for it. Also, first- 
year teachers are paid to attend the weeklong summer training offered to 
teachers new to the district. 

The four ESL instructional aides, aH high school graduates, provide 
instructional assistance to the students in the classroom. In addition, they 
administer the oral portion of the LAS language proficiency exam that 
must be given individuaUy to students. The ratio of students to aide is 7.5 
to 10 to one aide at the beginning level and 25.8 to one aide at the other 
two levels. Classroom/program aides receive an extensive daylong training 
at the beginning of the school year and teachers continue the training of the 
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aides assigned to them, 

A teacher shortage exists in all subjects. However, those teachers who 
have ESL endorsements may be given preference in hiring because of the 
high number of ESL students who are mainstreamed into the regular class- 
es. All teachers are paid according to the same pay scales. 

The recruitment consists of advertising in publications and sending offi- 
cials to various teacher fairs in Arizona and neighboring states. Also, a pro- 
gram is in place that encourages ESL aides to continue their formal educa- 
tion, The aides are reimbursed for college or university credits they earn 
that can be applied towards a teaching degree. Aides who have spent many 
years in the ESL classroom have become some of the best ESL teachers in 
the district. Another program has been piloted and will be offered next year 
in grades 11 and 12 to encourage students to go into teaching. 

Assessment Statistics 

All parents fill out a home language survey at the time of enrollment in 
accordance with federal regulations. Three questions are asked: 

■ What language did the student learn first? 

■ What language does the student most often speak? 

■ What language is spoken most often in the home? 

If any answer is not English, the student is classified as Primary Home 
Language Other Than English (PHLOTE) and then screened in order to 
find out if the student qualifies for LEP services. The score on the Stanford 
9 must be at or below the 40th percentile to remain eligible. After that has 
been ascertained, the student is assessed on the LAS English oral language 
proficiency test and a district writing proficiency test. 

Students are also interviewed in their primary language in accordance 
with state regulations, and LEP students receive dictionaries in the home 
language and English. Otherwise, they use the usual classroom textbooks. 

Once a year LEP students are reassessed in the areas of listen ing/speak- 
ing, reading, and writing to determine if they have attained English profi- 
ciency in one or more of those skills. The measuring of English language 
proficiency is accomplished by having each LEP student take the oral por- 
tion of the Language Assessment Scales (LAS) by CTB McGraw-Hill, a 
test that is administered one-to-one by an instructional aide. The LAS 
requires that each aide uses the cassette tape instead of his or her own voice 
to obtain standardized results. Reading assessments are done by means of 
the Stanford or the Stanford 9 (given to all students except LEP students 
who have been enrolled in Arizona schools fewer than three years). 
Students compose 125-word explanatory paragraphs to be used for the 
writing assessments. 

Evaluation of all students’ progress is accomplished by means of Stanford 
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9 scores, district criterion referenced tests (CRTs), and state standards (the 
AIMS test), portfolios (writing samples passed from the teachers of one 
year to those of the next year), and teacher observation. Teachers receive 
information concerning the students’ progress after the assessment data 
have been compiled. LEP Students average two years in the ESL program. 
However, many LEP students who enroll at Glendale High School are 
already at the intermediate or advanced levels. 

Exit criteria are based on English proficiency: speaking, reading, and 
writing. The students must score 4 or 5 on the LAS, read at 36 percent and 
above on the Stanford 9, and demonstrate writing ability in accordance 
with a district writing standard (the district writing assessment consists of 
a 125-word explanatory paragraph on a choice of topics). Students can 
enter and exit at any level. 

The district reclassification rate for 2000 was 9 percent while the state 
average was 8.8 percent. However, the graduating seniors are not figured 
into this tabulation. The school officials believe that the actual reclassifica- 
tion rate is much higher than what was reported to the Arizona 
Department of Education. 

The Stanford 9 Test is a standardized, nationally norm- referenced exam- 
ination administered annually to students in grades 2-12. LEP students 
who attended school in Arizona for fewer than three years have been 
exempt from taking this test. 



Table 1-6. 

Glendale High School 
Stanford 9 Percentile Rank Scores 



Grade School Year Pl996^155^pf ! 
Content > 

Area f>% i>5^Scor&AZ 


1997-1998 
% Score AZ 


1998TW 

%,,fSco#rAZ 


1999-2000 
% Score AZ 


9 Reading j 

Language 74^3^ % 37 

Mathematics \ 74?f"53'is iS4 


74 34 44 

75 32 39 

76 50 57 


92 s 32, 1-39.. 
92,1 Sili'157-; 


81 34 43 

83 3 4 40 

83 52 59 


10 Reading s42 

Language ^ 

Mathematics 


79 36 42 

79 40 43 

79 46 47 


1M‘ 43 49 


88 35 42 

87 37 44 

87 42 50 


11 Reading |72^ 

Language j72 45 42 

Mathematics [72 50 49 


78 39 46 

78 44 43 

78 50 51 


.ioq|44, 

iloo- -si IS 


77 43 45 

77 43 44 

77 49 55 



%= percentage of eligible student who took the test. Score = percentile rank score; 50 is the 
national percentile rank score. AZ= the percentile rank score of AZ students. 



The Arizona Instruments to Measure Standards (AIMS) Test is admin- 
istered annually at the high school level. When a student scores at least 500 
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points on any of the three sections of the test, that student has met the 
standard in that subject area. Table 1-7 below shows the spring of 2000 test 
score results. 



Table T7. 

Glendale High School 
AIMS Results 
School Year 1999-2000 



Grade 10 


Number 

Tested 


Mean 

Score 


FFB 


A 


M 


E 


Reading 


222 


522 


11% 


25% 


44% 


19% 


Writing 


222 


478 


15% 


53% 


32% 


0% 


Mathematics 


224 


453 


74% 


14% 


12% 


0% 


Grade 11 


Number 

Tested 


Mean 

Score 


FFB 


A 


M 


E 


Reading 


84 


507 


8% 


33% 


54% 


5% 


Writing 


131 


470 


15% 


63% 


22% 


0% 


Mathematics 


175 


454 


77% 


12% 


11% 


0% 


FFB = Fell far below the standard. A = Approaching the standard, 
M = Met the standard E = Exceeds the standard. 







Thunderbird High School 
LEP Program 

English Acquisition Program — 

Grades 9-12 

Johanna Haver 

Program Elements 

Thunderbird High School is one of nine comprehensive secondary high 
schools in the Glendale High School District. One principal and three 
assistant principals oversee 87 teachers, seven other professional staff, and 
eight teacher aides. Forty-three of those teachers have a Master’s Degree or 
higher; forty- four of them have taught for ten years or more. The enroll- 
ment of Thunderbird High School is about 1780 students with approxi- 
mately 147 identified as Limited English Proficient (LEP). 

Two hundred students earned AP credits in classes that include English 
literature, French, Spanish, U.S. history, U.S. government, physics, and cal- 
culus during the 1999-2000 school year. The district pays for the AP test- 
ing of all students. During the 1999-2000 school year, Thunderbird stu- 
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dents received special recognition for National Merit Scholarships, a supe- 
rior rating for its jazz band, and state/regional athletic championships. 

LEP students received special recognition for activities that they partic- 
ipated in during the 1999-2000 school year. The Arizona School Public 
Relations Association presented an award to Thunderbird High School for 
Thunderbird Foreign, a newspaper that is made up of stories by the LEP 
students. This 20-page newspaper is mailed to families of LEP students 
and distributed to all Glendale Union high schools. Also, the Thunderbird 
Culture Club was awarded the Glendale Union High School District annu- 
al “Achievement above All” distinction. 

Remedial help is available to students. The school offers extensive aca- 
demic assistance to all students before and after school with peer mediation 
and a peer-tutoring program also in operation, A one-credit class has been 
established at Thunderbird HS to train students in peer tutoring. The ESL 
resource classes provide an ESL teacher, an instructional aide, sometimes 
computers, and peer-tutors who are either advanced or former LEP stu- 
dents. 

Thunderbird High School uses a structured English immersion model 
for LEP instruction. The philosophy is that LEP students need to be taught 
English intensively to such a degree that they can move as soon as possible 
into the mainstream to participate in all the academic opportunities of 
Thunderbird High School. 

It is the policy of Thunderbird High School to explain to the parents of 
students identified as LEP that participation in the ESL program is volun- 
tary, There is no sheltering outside of reading and English classes. ESL stu- 
dents are encouraged strongly by ESL teachers, content-area teachers, and 
administrators to participate in the many school activities. 

The ideal case progression of an LEP student’s education occurs when 
that student has literacy in his or her native language. A literate, well-edu- 
cated child who speaks no English initially can move quickly through the 
ESL program in two years, and, after that, possibly qualify for honors 
and/or Advanced Placement (AP) calculus. 

The worse case progression occurs when an LEP student has received no 
formal education. It is very difficult for any high school to make up for the 
lost time, especially when the instruction is in a language unfamiliar to the 
student. Generally, such students need a great deal of remedial assistance. 

English as a Second Language (ESL) classes are offered in grades 9-12 
to students according to their particular level of English language profi- 
ciency. 

Beginning Level — 4 periods per day — 15 students to one ESL endorsed 
teacher (20 max) and two aides 
ESL Speaking 
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ESL Reading 
ESL Writing 
ESL Teens 

Intermediate Level — 2 periods per day — 25.8 students to one ESL 
endorsed teacher (same as regular class) and one aide 
ESL English 1-2 
ESL Reading 1-2 

Advanced Level — 2 periods per day — 25.8 students to one ESL endorsed 
teacher (same as regular class) and one aide 
ESL English 3-4 
ESL Reading 3-4 

ESL Resource — 1 period per day — 25.8 students to one ESL endorsed 
teacher (same as regular class) and one aide 

Beginning ESL Speaking. The instruction concentrates on building 
basic listening, speaking, reading, and writing skills. Students are taught 
how to pronounce words clearly, follow oral directions and commands, and 
to respond appropriately to oral questions. The students learn to use target 
vocabulary and simple grammatical forms correctly while speaking on a 
variety of topics. This includes spontaneous conversation, prepared dia- 
logues, and speeches. 

Beginning ESL Writing. Beginning students are taught simple and 
compound sentence structure, usage, and mechanics. The correct use of tar- 
get vocabulary in context is emphasized. Composition pieces introduced 
include a brief personal narrative and/or poem, short summary or factual 
information, personal letter, brief personal information card, simple appli- 
cation, and explanatory paragraph. 

Beginning ESL Reading. Students develop oral and silent reading 
skills. While reading fiction and non-fiction, students practice reciprocal 
reading strategies, which is the process of reading paragraphs orally in pairs 
and then asking each other questions about those paragraphs. Extensive 
vocabulary, selected literary terms, and selected literary elements are intro- 
duced. Students are taught to analyze both fiction and non-fiction for over- 
all meaning. 

Teens. Due to the highly diversified ESL population at Thunderbird 
High School, it became necessary to offer beginners a fourth-hour class 
that serves as a bridge to understanding and adjusting to American culture. 
The curriculum includes subjects such as personal hygiene, table manners, 
appropriate clothing, rules for sports, restaurant menus, school regulations, 
taking the bus, and recreational possibilities. 

During the rest of the day the beginning students are mainstreamed into 
classes that do not require print-based material. They include math, PE, 
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art, and music. 

Intermediate Level ESL English 1-2. Students develop English v^iting 
proficiency along with effective oral presentation skills and reading strate- 
gies. The composition pieces include the explanatory essay, summary, per- 
sonal narrative, and personal letter. Students learn to apply correct conven- 
tions to their writing and to locate resources in the media center. The cur- 
riculum is based on the 9th grade language arts curriculum. 

Intermediate Level Reading 1-2. Development of reciprocal reading 
strategies for both fiction and non-fiction is encouraged. This is a process 
proven highly effective in which pairs of students read orally back and forth 
to each other, one paragraph at a time, and then ask questions of each other. 
The study of literature is introduced by reading short stories, poetry, nov- 
els, and/or plays. Students are taught to analyze the short story and poetry 
for literary elements, figurative language, and overall meaning. Oral and 
written communication skills are included. 

During the rest of the day, the intermediate students are mainstreamed 
into history and introductory science as well as math, PE, art, and/or music. 

Advanced Level ESL English 3-4. Similar to English 1-2, but present- 
ed at a higher level and with more content. The curriculum is based on the 
10th grade language arts curriculum. 

Advanced Level ESL Reading 3-4. Similar to Reading 1-2, but pre- 
sented at a higher level and with more content. 

During the rest of the day, the advanced students take such classes as 
U.S. history, biology, and computer class. No class is closed to these stu- 
dents and many take advanced-math classes, honors classes, and Advanced 
Placement (AP) calculus. 

ESL Resource. This class consists of an ESL teacher, an instructional 
aide, and peer tutors to provide academic support for LEP students in 
mainstream classes. Study skills and test taking strategies are taught, and 
access to computers is available at selected sites. 

One to two school buses (depending on the number of students and 
schools involved) adjust their schedules and routes to pick up and return 
ESL students to those Glendale Union high schools that do not have ESL 
programs. Those students can then participate in the ESL program at 
Thunderbird. When those students exit the ESL program, they have the 
choice to either remain at Thunderbird or to return to the original school. 

All LEP students are encouraged to take pre- algebra at summer school 
for six weeks. This is available to all students. ESL classes are offered at 
summer school to LEP students who enrolled as beginners late in the 
school year and/or need to work on literacy. This lasts for five to six weeks 
at one campus, but is open to LEP students from all of the district schools. 
The ESL students are not charged; separate district funding is available for 
this. 
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LEP Statistics 

Approximately half of the entering LEP students in the entire district come 
from feeder schools and the other half come from other countries. Total 
number of LEP students equaled 130 for the 1999-2000 school year and 
147 for the 2000-2001 school year. 



Table 1-8. 

Thunderbird High School 

Distribution of Students According to Grade and ESL Class 
School Year 2000-2001 



Grade Level 


9th grade 




■miTggr.M 




Classes 


Sections 


Beginning 
4 hrs per class 


22 


14 


5 


1 


3 


12 


Intermediate 
2 hrs per class 


11 


16 


12 


3 


2 


4 


Advanced 
2 hrs per class 


13 


13 


15 


10 


2 


4 


Resource 1 hr 


6 


4 


10 


5 


1 


1 


# of students 


52 


47 


42 


19 


8 


21 



The total numbers on the table above exceed the total of LEP students due to an over- 
lap with the resource groups. The classes of students are the groups that meet together 
for 1-4 hours each day. Each of those hours counts as a separate section. Thus, eight 
classes, or groups, attend 21 sections. The eight groups are served in 21 sections. 



For the 1999-2000 school year, 59 students spoke Spanish (approxi- 
mately 45 percent). Two students spoke Albanian, two students spoke 
French, one student spoke Kurdish, nine students spoke Vietnamese, seven 
students spoke Chinese, two students spoke Dinka, nine students spoke 
Arabic, one student spoke Lao, one student spoke Polish, thirty-four stu- 
dents spoke Serbo-Croatian/Bosnian, one student spoke Punjabi, one stu- 
dent spoke Tagalog, and one student spoke Persian. 

For the 2000-2001 school year, 84 speak Spanish (approximately 57 per- 
cent). Three students speak Albanian, one student speaks French, six stu- 
dents speak Vietnamese, one student speaks German, four students speak 
Chinese, nine students speak Arabic, two students speak Polish, thirty-one 
students speak Serbo-Croatian/Bosnian, one student speaks Punjabi, two 
students speak Tagalog, one student speaks Assyrian, and two students 
speak “other.” 

Language Line from Monterey provides 24-hour interpreters for teacher 
to use a 3-way conversation in which the language of the parent or guardian 
is quickly translated. However, it costs $2.50 per minute, and teachers are 
asked to keep the conversations short. Family literacy classes are offered to 
parents at Glendale HS and funded through a Tide-1 family literacy pro- 
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gram. 

No students receive either reduced price or free school lunch. However, 
students may work in the cafeteria for pay and free meals. According to data 
collected from the students during the 1999-2000 school year, the follow- 
ing work patterns exist among all students at Thunderbird HS: 3 percent 
have full-time jobs; 43 percent have part-time jobs; and, 3 percent have 
both full-time and part-time jobs. The school officials believe that LEP 
students are considerably more likely to work than other students. 

Staffing Model 

The student to teacher ratio for LEP students at the beginning level is from 
15 to 20 students to one ESL teacher; the intermediate and advanced lev- 
els have the same ratios as exist in mainstream classes, 25.8 to one teacher. 
All five teachers of ESL classes hold ESL endorsements. 

AH teachers serve LEP students. And while the content-area teachers’ 
backgrounds in ESL vary, all content-area teachers have access to ESL in- 
service training and they are encouraged, but not required, to have ESL 
endorsements. 

Each school also has a “mentor” teacher who helps aU 1st, 2nd, and 3rd 
year teachers — regardless of a new teacher’s former experience. That men- 
tor teacher has three class periods free to instruct the teachers concerning 
classroom management, cooperative learning, Effective Elements of 
Effective Instruction (EEEI), and other teaching strategies that are helpful 
to aU teachers. This districtwide mentoring program has been recognized 
nationaUy. 

Special ESL workshops are provided to aU teachers throughout the 
school year. This is arranged through Karen Henderson, the ESL coordi- 
nator (a 3/5 rather than fuH-time position). She is often asked to teach 
strategies designed especiaUy for LEP students to a team of content-area 
teachers. Furthermore, as teachers practice these strategies, they share what 
works with their coUeagues. 

Both non-ESL and ESL endorsed teachers receive a lot of cross-devel- 
opment specialized staff development. The teachers who wish to attend 
summer staff development are paid for it. Also, first-year teachers are paid 
to attend the weeklong summer training offered to teachers new to the dis- 
trict. 

The six ESL instructional aides, aU high school graduates, provide 
instructional assistance to the students in the classroom. In addition, they 
administer the oral portion of the LAS language proficiency exam that 
must be given individuaHy to students. The ratio of students to aides is 7.5 
to 10 to one at the beginning level and 25.8 to one at the other two levels. 
Classroom/program aides receive an extensive daylong training at the 
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beginning of the school year, and teachers continue the training of the aides 
assigned to them. 

A teacher shortage exists in all subjects; however, those teachers who 
have ESL endorsements may be given preference in hiring because of the 
high number of ESL students who are mainstreamed into the regular class- 
es. All teachers are paid according to the same pay scales. 

The recruitment consists of advertising in publications and sending offi- 
cials to various teacher fairs in Arizona and neighboring states. Also, a pro- 
gram is in place that encourages ESL aides to continue their formal educa- 
tion. The aides are reimbursed for college or university credits they earn 
that can be applied towards a teaching degree. Aides who have spent many 
years in the ESL classroom have become some of the best ESL teachers in 
the district. Another program has been piloted and will be offered next year 
in grades 11 and 12 to encourage students to go into teaching. 

Assessment Statistics 

All parents fill out a home language survey at the time of enrollment in 
accordance with federal regulations. Three questions are asked: 

■ What language did the student learn first? 

■ What language does the student most often speak? 

■ What language is spoken most often in the home? 

If any answer is not English, the student is classified as Primary Home 
Language Other Than English (PHLOTE) and then screened in order to 
find out if the student qualifies for LEP services. The score on the Stanford 
9 must be at or below the 40th percentile to remain eligible. After that has 
been ascertained, the student is assessed on the LAS English oral language 
proficiency test and a district writing proficiency test. 

Students are also interviewed in their primary language in accordance 
with state regulations and LEP students receive dictionaries in the home 
language and English. Otherwise, they use the usual classroom textbooks. 

Once a year LEP students are reassessed in the areas of listening/speak- 
ing, reading, and writing if they have not yet attained English proficiency 
in one or more of those skills. The measuring of English language profi- 
ciency is accomplished by having each LEP student take the oral portion of 
the Language Assessment Scales (LAS) by CTB McGraw-Hill, a test that 
is administered one-to-one by an instructional aide. The LAS requires that 
the aides use the cassette tape instead of their own voices to obtain stan- 
dardized results. Reading assessments are done by means of the Stanford 9 
(given to all students except LEP students who have been in Arizona fewer 
than three years). Students compose 125 -word explanatory paragraphs to 
be used for the writing assessments. 

Evaluation of all students’ progress is accomplished by means of Stanford 
9 scores, district criterion referenced tests (CRTs), and state standards (the 
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AIMS test), portfolios (writing samples passed from the teachers of one 
year to those of the next year), and teacher observation. Teachers receive 
information concerning the students’ progress after the assessment data 
have been compiled. 

Students average two years in the program; however, many LEP students 
who enroll at Thunderbird High School test already at the intermediate or 
advanced levels. Exit criteria are based on English proficiency: speaking, 
reading, and writing. The students must score 4 or 5 on the LAS, read at 36 
percent and above on the Stanford 9, and demonstrate v^iting ability in 
accordance with a district writing standard (the district waiting assessment 
consists of a 125-word explanatory paragraph on a choice of topics). 
Students can enter and exit at any level. 

The district reclassification rate for 2000 was 7 percent while the state 
average was 8.8 percent. However, the graduating seniors are not figured 
into this tabulation. The school officials believe that the actual reclassifica- 
tion rate is much higher than what was reported to the Arizona Depart- 
ment of Education. 

The Stanford 9 and the ACT scores were above district, county, state, 
and national averages in all grade levels and in all subjects for the 1999- 
2000 school year. 

The Stanford 9 Test is a standardized, nationally norm-referenced exam- 
ination administered annually to students in grades 2-12. LEP students 
who attended schools in Arizona for fewer than three years have been 
exempt from taking this test. 



Table T9. 

Thunderbird High School 
Stanford 9 Percentile Rank Scores 



Grade School Year 
Content 
Area 


1997-1998 

'Scofe^*^^ % Score AZ 




1999-2000 
% Score AZ 


9 Reading 

Language 
Mathematics 


f- 87.^ - 54- ,'-^3 1 90 54 44 

i8‘8fi5^W3y 89 52 39 

89 71 57 


100 55 ' 43. 

100 52 , 39' 

100 74 57 


84 57 43 

85 54 40 

81 73 59 


10 Reading 
Language 
Mathematics 


lf#^5Sl!;S42si 81 57 42 

81 58 43 

mmmm so 6 i 47 


IKi 


83 59 42 

83 59 44 

82 65 50 


11 Reading 
Language 
Mathematics 


86 60 46 

p8i#58jfM 86 55 43 

irg8l"®1fP9l 86 66 51 




84 54 45 

82 51 44 

81 66 55 



%= percentage of eligible student who took the test. Score = percentile rank score; 50 is the 
national percentile rank score. AZ= the percentile rank score of AZ students. 



The Arizona Instruments to Measure Standards (AIMS) Test is admin- 
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istered annually at the high school level. When a student scores at least 500 
points on any of the three sections of the test, that student has met the stan- 
dard in that subject area. The table below shows the spring of 2000 test 
score results. 

Table I-IO. 

Thunderbird High School 
AIMS Results 
School Year 1999-2000 



Grade 10 


Number 

Tested 


Mean 

Score 


FFB 


A 


M 


E 


Reading 


383 


551 


4% 


9% 


48% 


38% 


Writing 


378 


505 


4% 


36% 


59% 


1% 


Mathematics 


379 


473 


60% 


14% 


24% 


2% 


Grade 11 


Number 

Tested 


Mean 

Score 


FFB 


A 


M 


E 


Reading 


34 


474 


50% 


24% 


21% 


6% 


Writing 


49 


474 


27% 


33% 


41% 


0% 


Mathematics 


67 


465 


58% 


24% 


18% 


0% 



FFB = Fell fer below the standard, A = Approaching the standard. 
M = Met the standard E = Exceeds the standard. 



Bethlehem Area School District — 
Bethlehem, PA 

English Acquisition Program K-5 

Rosalie Pedalino Porter 

Program Background 

The Bethlehem Area School District in northeastern Pennsylvania draws 
its 14,000 students from the City of Bethlehem (60 percent), adjacent sub- 
urbs (30 percent), and the rest from nearby rural districts. The Bethlehem 
area has a substantial unemployment problem due to the decline of manu- 
facturing, with approximately 25 percent of the public school students com- 
ing from families of low socioeconomic status. The ethnic/racial composi- 
tion of the student body is 25 percent Hispanic, 4 percent African Ameri- 
can, and 1 percent Asian, and the remainder classified White-non-Hispan- 
ic. 

The Bethlehem schools enroll approximately 1,400 students of limited- 
English proficiency (LEP) for whom English Acquisition Program servic- 
es are provided. These students are predominantly Spanish speakers (86 
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percent) from Puerto Rico and from Central American countries, while the 
remaining 14 percent represent a dozen other language backgrounds. The 
present study is focused on the LEP students in the 14 elementary schools, 
grades K-5 , which constitutes the largest segment of the English language 
learners in the Bethlehem schools. 

LEP Population 

The Bethlehem program for LEP students, first implemented in 1993, fits 
under the broad definition of Structured English Immersion. The philoso- 
phy of the Bethlehem program is based on the belief that providing LEP 
students with fluency and literacy in English as rapidly as possible promotes 
the best opportunities for academic achievement in the mainstream class- 
room and for future success in the larger community. The approach 
employed in the elementary program follows: 

1. Providing an academic setting that accelerates the acquisition of English 
language skills for academic and social purposes. 

2. Integrating LEP students with native speakers of English as much as 
possible. 

3. Coordinating support services (cooperative planning by classroom and 
ESOL teachers. Title I, etc.) to avoid or reduce fragmentation of the 
school day. 

4. Using the student’s native language for brief explanations, when neces- 
sary. 

Special approaches employed in the elementary schools include the fol- 
lowing: 

1. There is reduced class size in schools or grade levels where the enroll- 
ment of LEP students is higher than 30-50 percent of the total. 

2. All LEP students attend their local school and are assigned to main- 
stream classrooms, with special English Acquisition Program instruction 
by trained ESOL teachers, according to their level of English on arrival, 
i.e., LEP students in the Beginner category (little or no English) receive 
75 minutes daily of ESOL; Intermediate level students receive 45- 
minute lessons; and Advanced level students receive special instruction 
based on their particular needs. 

3. Beginner level students may be assigned individual tutors (called Second 
Language Guides) for 5 hours/week, in addition to their special ESOL 
classes. 

4. Itinerant ESOL teachers provide small-group instruction 2-4 times per 
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week for LEP students in low-impact schools (less than 10 percent LEP 
enrollment). 

5. Kindergarten LEP students are not given special ESOL lessons as the 
entire program at this level focuses on language development and on pre- 
literacy and pre-math skills. A special grant provides for an additional 
hour of Kindergarten beyond the district's half-day schedule for high 
impact LEP schools. 

Participation in the English Acquisition Program is dependent on 
parental approval, once the program has been explained to parents. Parents 
or students (high school level) may sign a waiver declining program servic- 
es. In eight years only ten waiver requests have been submitted. 

The elementary program for LEP students requires that they participate 
in the mainstream curriculum in all subject areas, with adaptations for their 
level of English. ESOL instruction is based on the Content-Based 
Language Teaching approach which emphasizes the use of school subjects 
as the focus of English language lessons, promoting second language learn- 
ing and content learning simultaneously. 

English language skills are taught through a balanced literacy framework 
which develops listening comprehension, speaking, reading, and writing 
concurrendy. Balanced literacy instruction for LEPs employs these various 
techniques to support the reading lessons in the mainstream classroom: 
guided reading, shared reading, interactive writing, guided writing, and 
word work (phonemic awareness and phonics), ESOL teachers also focus 
on helping students to meet the reading and writing standards set by the 
State of Pennsylvania and the New Standards adopted by the Bethlehem 
Area School District. 

Ideally, the LEP students at one end of the spectrum are those who have 
received a good education in their land of origin and are up to grade level 
in their knowledge of content and literacy in their native language. For 
these students, the rapid acquisition of English and the transfer of subject 
matter knowledge from their first language to English may be accom- 
plished in only one or two years. Special difficulties arise in the normal pro- 
gression of the English Acquisition Program for LEP students who arrive 
in Bethlehem at the upper elementary grades or the secondary level having 
missed several years of schooling in their land of origin. For older students 
who have little or no knowledge of English and are also lacking subject 
matter knowledge, the challenge to help them catch up with their age/peer 
group is a very difficult one. 

LEP students are assessed twice a year, in November and June, to chart 
progress in English language skills development.^ The English language 

A detailed description of the stages of language development can be found in READ 
Perspectives^ Vbl. II-2, pp. 116-119. 
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curriculum has been revised periodically to bring it into closer aUgnment 
with the Bethlehem New Standards for all students. The last formal pro- 
gram review was done in 1996. The English Acquisition Program is cur- 
rently under review by a committee of eleven principals from elementary, 
middle, and high school levels and the program coordinator. 

It should be reiterated here that LEP students at the elementary level are 
not served in a separate classroom but are assigned to a mainstream class- 
room from the first day, with special ESOL support part of each day. The 
length of time any student needs to receive support services is determined 
by the level of English proficiency of each student (as well as previous edu- 
cational background and other factors) at the time of enrollment in the 
Bethlehem schools. LEP students arrive at different age levels, with differ- 
ent abilities and with different levels of English language proficiency. In 
general, students arriving with little or no English will be exited from pro- 
gram services in three to four years. Students with strong academic back- 
grounds may exit the program in one year; students with learning disabili- 
ties or other problems will need more time and a variety of services besides 
ESOL. 

Eighty- six percent of the LEP students are Spanish speakers of Puerto 
Rican ethnicity. Since Puerto Rico is part of the U.S., place of birth is not 
separately coded. It is safe to say that most all of these students were born 
on the U.S. mainland of Spanish-speaking families that move frequently 
and travel back and forth to Puerto Rico, sometimes for extended stays. Of 
aU the Spanish- speaking students, less than 10 percent are from Central 
and South America and arrive in the U.S. with no English. Fourteen per- 
cent of the LEP students who are not Spanish speakers come from a dozen 
different countries where English is not the common language. 

The district provides translations of major documents (report cards, 
newsletters) in Spanish and English but not in the native languages of the 
smaller language groups. Some grant money has been provided for adult lit- 
eracy, ESL, and parent/child literacy classes through certain schools. 

Other special program services include: 

1. ESOL summer school for elementary school students at the Beginner 
level provides 5 hours daily for four weeks, and is taught by ESOL teach- 
ers. 

2. There are after-school programs through special grants and migrant edu- 
cation funds. 

3. Reading Recovery Program had been in operation in Bethlehem schools 
for eight years. It provides 12-20 weeks of intensive one-on-one reading 
instruction to first graders who score in the lowest 20 percent of their 
class. The majority of students who receive Reading Recovery assistance 
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go on to read at the average level for their grade and require no further 
remedial services. 

LEP Statistics 

The elementary schools in Bethlehem enroll 760 LEP students, though this 
number fluctuates throughout the school year due to the high rate of tran- 
sience of district families. Eighty-six percent of the LEP students are 
Spanish speakers, mainly from Puerto Rico with small numbers from 
Mexico, Guatemala, Colombia, and El Salvador. The others are speakers of 
(in order of frequency): Chinese dialects, Portuguese, Greek, Arabic, 
Turkish, Vietnamese, French, Russian, Croatian, Albanian, Swedish, 
Gujarati, Korean, Punjabi, Thai, and Urdu. 



Table I-ll. 

Bethlehem Area School District 
LEP Students Receiving ESOL Services 



Grade 


Beginner 


Intermediate 


Advanced 


Special Ed. 


Elxited 


K 


60 


24 


6 


2 


0 


1 


91 


46 


12 


7 


11^ 


2 


66 


42 


19 


1 


13- 


3 


35 


47 


21 


11 ' 


37' 


4 


33 


49 


29 


19 


47 


5 


23 


43 


38 


14 


53 


Source: Bethlehem Area School District 



Note: Many of these students transferred into the district in the upper grades and many of the 
Special Ed students were alreac^ identified with Individual Education Plan (lEP) from their 
sending district. 



The number of Bethlehem students receiving free or reduced cost lunch 
(as an indicator of family poverty) is estimated to be 25 percent. As many 
as 80 percent of children attending high and medium impact schools are 
likely to receive free/reduced cost meals. 

Although this report does not extend to the high school level, it is a fact 
that most LEP high school students work after school in an English-speak- 
ing enwonment. Their work often provides necessary income for their 
families. Many of the Bethlehem LEP students come from families that 
have moved frequentiy, and have experienced serious dislocations due to 
unsettled conditions, illnesses, and incarceration. Some of the children live 
with relatives rather than with parents. 

Staffing Model 

Specialist ESOL teacher/student overall (18 full time teachers for 750 LEP 
students) is roughly 1/41. Because LEP students are assigned to main- 
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stream classrooms with their English-speaking peers for a good portion of 
the school day, ESOL teachers deliver instruction to groups of from 3 to 14 
students at a time, depending on the English language level of the students, 
their grade level, and the numbers of LEP students in the school. Lessons 
may be given in a pull-out setting or in a “push-in” situation (ESOL teacher 
teaches a small group vsdthin the classroom.) In addition, a full-time tester 
(a professional position) is on staff at the Center for Language Assessment, 
the office of the Program Coordinator where aU new students are inter- 
viewed and tested for enrollment in the Bethlehem schools, 

Pennsylvania does not have a bilingual education mandate and does not 
have a certification for bilingual or ESOL teachers. The only requirement 
is elementary or secondary teaching certification. The Bethlehem district 
has retained most of the Spanish bilingual teachers from the former bilin- 
gual program, and with retraining, they are staffing most of the ESOL 
positions, in addition to new teachers who have been recruited in recent 
years. 

Non-specialist teachers serving LEP students are the regular elementary 
level classroom teachers, but they are not considered part of the program. 
To reiterate the basic philosophy of the program, LEP students -will bene- 
fit the most from participating in a regular classroom with English-speak- 
ing children, while receiving support services from the ESOL staff. 

A rich array of staff development workshops — in-service training — ^has 
been provided for teachers of English Language Learners (the newest 
appellation for LEP students), including the follovsdng: 

1. Balanced literacy instruction — as previously discussed — is provided. 

2. Integrated Language Approach workshops by Dr. George Gonzalez on 
techniques for using classroom literature texts to build oral language, 
reading, and writing are available. 

3. ESOL teachers are trained in implementing the Developmental Reading 
Assessment (DRA) to evaluate student progress in oral reading and com- 
prehension. They have established the benchmark levels for determining 
when LEP students move from Beginner to Intermediate to Advanced 
status. 

4. Reading Recovery Teacher Leaders are trained for a full year for provid- 
ing student tutorials and for training district teachers in Reading 
Recovery and balanced literacy approaches. 

Teacher aides are rarely used in the Bethlehem English Acquisition 
Program. Teacher aides, who are designated Second Language Guides, are 
employed for small group tutoring of Beginner level students in schools 
with very small enrollments of LEP students where a full-time ESOL 
teacher is not feasible. Second Language Guides also provide support at 
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two Title 1 -funded elementary schools. In all cases, they work closely with 
ESOL teachers and receive direction on methods and materials from them. 
The aides are sometimes included in the district in-service training days. At 
present, there are two fuU-time equivalent teacher aides in the English 
Acquisition program working in Kindergarten classes with high enroll- 
ments of LEP students. 

In-service training is offered to all teachers who work with LEP students 
and is encouraged by school principals. The majority of elementary school 
regular classroom teachers have not attended these workshops, but they 
have been trained in the balanced literacy approach and in the techniques 
for teaching the New Standards, both of which work very well vwth LEP 
students. 

ESOL teachers regularly attend meetings and in-service days to discuss 
program improvements, assessment methods, and teaching strategies. 
These teachers developed the rubric for oral reading and comprehension 
testing used to determine when students have reached a higher level of 
English proficiency and when they are ready to exit the English Acquisition 
Program. They are now working on writing instruction and writing tests to 
meet the four types of writing required in the New Standards. New ESOL 
teachers receive in-classroom coaching by the Program Coordinator on lit- 
eracy strategies and assessment techniques used in the program. Summer 
school ESOL teachers are given two days of special training by the Reading 
Recovery Teacher Leaders who then make follow-up visits to the classroom 
for observing/coaching. 

The Bethlehem Area School District enjoys a reputation as an outstand- 
ing district and receives many applications for each teaching position. The 
Program Coordinator has recruited many very competent new teachers 
who are certified at the elementary level and are fluent in Spanish and 
English, from across the country. The Coordinator maintains an active file 
of teacher candidates, conducts telephone interviews of likely applicants, 
and keeps an eye out for teachers who may be interested in entering the 
ESOL field. No additional pay or financial inducements are given to ESOL 
teachers. The standard recruitment methods used by seasoned administra- 
tors are employed here, i.e., networking with other program coordinators, 
attending professional conferences, and canvassing local universities gradu- 
ate programs. 

As for retention of staff, there is very litde turnover in the ESOL teach- 
ing positions that are full-time and assigned to one school. More ESOL 
teachers who are part-time or who have assignments as itinerant teachers 
tend to leave when a better position becomes available. 

The academic progress of LEP students is monitored by periodic assess- 
ments of language proficiency through DRA and other measures, report 
card grades, and consultations with regular classroom teachers, in order to 
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determine when students are ready to exit the English Acquisition Program 
and participate in mainstream classroom activities without its support. 
Teachers are the main arbiters of student success based on mastery of stages 
in oral language, reading, and writing. In occasional cases, teachers may call 
for additional student assessment by the tester from the central office. 

ESOL teachers employ texts, manipulatives, dictionaries, visual aids, 
computer programs, videos, and teacher-made materials in their instruction 
of LEP students. In the elementary schools, the district curriculum in all 
subjects is the curriculum that is adapted for the LEP students. The over- 
arching goal of the Bethlehem Area District Schools is student success in 
the regular program. 

Assessment Statistics 

Bethlehem is in the process of adopting a standardized exam to administer 
to all smdents and so no such achievement data are available. However, data 
on student assessment are in a comprehensive database, although not by 
grade level. Two- thirds of the students who enter the Bethlehem schools at 
the Beginner level of English proficiency will have exited the English 
Acquisition Program after 3-4 years. 

Data reported in a study of the Bethlehem program published in the fall 
of 1995^ record LEP student achievement at the end of the first two years 
of the program as follows: 

As of June, 1995, analysis of data from all students in the program, grades K- 
12 showed that 29 percent of students at the beginning level in the first quar- 
ter of the school year had moved to the intermediate level by June. Twelve 
percent of the students who were in the intermediate level the first quarter of 
the year had moved to the advanced level by June. Forty-sbc percent of the 
students who were classified as advanced in November had qualified to exit 
the program by June. 

In a four-year longitudinal report published in READ Perspectives,"^ Beth- 
lehem reports LEP student progress in English language proficiency as fol- 
lows: 



^READ Perspectives, Vol, II-2, p. 108. 

*READ Perspectives, Vol, V-2, Fall 1998, p. 70. 
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Table 1-12. 

Bethlehem Area School District 
K-5 Longitudinal Data 
Classes Entering 1994-95 and 1993-94 

Class Entering 1994-95 ' Class Entering 1993-94 



198 Students Progress 179 Students Progress 

At End of 3 Years At End of 3 Years 





Number 


Percent 


Number 


Percent 


Students Exiting 


64 


32 


100 


56 


Advanced Level 


48 


24 


30 


17 


Intermediate Level 


51 


26 


28 


16 


Beginner Level 


14 


7 


8 


4 


Special Education 


21 


11 


13 


7 


Source: READ Perspectives j Vol. V-2, Fall 1998, p. 70. 



Houston Independent School District 
RITE/Rodeo Program 

Russell Gersten 
Eugene Research Institute 

Program Description 

The RITE/Rodeo Program is a structured English immersion program. 
Students are taught entirely in English beginning in Pre-Kindergarten. 
They acquire English language ability as they learn to read in English, and 
discuss what they read with teachers and peers. 

The program uses a direct instruction approach.^ This approach has a 
history of proven success in building language and reading ability. This suc- 
cess was documented in the independent evaluation of Project Follow 
Through‘S as well as in other research documenting the effectiveness of this 
approach with English language learners.^ Critical features of direct in- 
struction are: frequent oral responses, group (choral responding) prior to 
individual responses, cumulative review, frequent feedback to students on 
accuracy, teaching so that there is high level of student success, and explic- 
it work on word attack strategies. 

The basic curriculum is the SRA Reading Mastery series (an updated 

'Becker, 1977; Carnine, 1991; Gersten, Woodward, 8c Darch, 1986. 

‘Stebbins, St Pierre, Proper, Anderson, 8c Cerva, 1977. 

decker 8c Gersten, 1982; Gersten, 1985; Gersten 8c Woodward, 1985. 
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version of the Distar program used in Project Follow Through) and the 
SRA Language for Learning Program (an updated version of Distar 
Language used in Follow Through). These are updated, refined versions of 
the curriculum used in the evaluation research described in the previous 
paragraph. 

The fact that the reading curriculum uses a carefully controlled vocabu- 
lary makes it an ideal vehicle for an immersion program. In addition, the 
teachers supplement the program with sheltered English techniques. For 
example, teachers use concrete objects as a means of teaching vocabulary 
and as a way to spend more time preteaching and practicing vocabulary 
than would be typical 'with native English speakers. 

Teachers use the oral language program as a springboard for “boardwork” 
(i.e., written language activities). Furthermore, students are actively encour- 
aged to read a wide range of books to increase their knowledge of English. 
Instruction is systematic and step-by-step. Rates of teacher-student inter- 
action are extremely high, students are explicidy taught relevant phonics 
and phonemic skills in reading, and formal academic language is explicidy 
taught. The key philosophy is that every child can learn. No assumptions 
are made about background knowledge. 

Characteristics of Students Served 

This program is implemented in 20 low-income schools. Eight of these 
schools have populations of English language learners. The program is used 
from Pre-K to second grade. 

Sutton and Bonner Elementary are two of the key schools with high 
numbers of Hispanic students who are English-language Learners. Table I- 
13 presents student enrollment and free lunch status. Table 1-14 presents 
demographics of the teaching faculty. Table 1-15 gives achievement profiles 
in both English and Spanish for grades 1 and 2. 



READ PERSPECTIVES 

46 





Table 1-13. 

RITE/Rodeo Progiam 
Enrollment Data and Free Lunch Status 
Two Selected Elementary Schools 





Bonner Elementary 




Enrollment 


Total 


831 


1,352 


Kinderg;arten and Pre-K 


132 


339 


Grades Served 


K-5 


EE-5 


Students by Program 


Bilingual Students 


54% 


51% 


ESL 


4% 


18% 


Limited English (LEP) 


58% 


72% 


Free/Reduced Lunch 


96% 


91% 




Table 1-14. 






RITE/Rodeo Program 




Teacher Profile at Two Elementary Schools 




Bonner Elementary 




Years of Experience 


5 or less 


38% 


49% 


6 to 10 


31% 


12% 


11 or more 


31% 


39% 


Teachers by Program 


Regular 


29% 


30% 


Bilingual/ESL 


55% 


51% 


Special Education 


17% 


12% 


Advanced Degrees 


Masters 


33% 


15% 
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Table 1-15. 

RITE/Rodeo Program 

Schoolwide Student NCE Score and Percentile 



Bonner Elementary 



Sutton Elementary 



97-98 




97-98 



98-99 




Score 



Score 






SAT 9 

(Reading in English) 






Grade 1 



41 



33 



47 



45 




Grade 2 



48 



46 




43 



37 



SAT 9 

(Math in English) 



Grade 1 



31 



19 



43 



37 



Grade 2 



43 



37 



mm 38 



29 



Aprenda 

(Spanish Reading) 






Grade 1 



55 



60 



.. 00 
oZ 



49 



47 



Grade 2 



60 



68 



o/: 



mm 



55 



60 



Aprenda 




Grade 1 


40 


32 




59 


67 




Grade 2 


54 


58 




62 


72 





Entry/Elxit Criteria 

Students are typically in the program from Pre-K to second grade. Pre-K is 
half day, and K, 1, and 2 are full day programs. Occasionally a student will 
remain in the program for a portion of the third grade until he or she com- 
pletes the SRA Reading Mastery Program or the language arts program. 
The average length is three years if preschool is included. 

The schools are all neighborhood schools. All students in the school (be 
they native English speakers or English-language learners) are in the RITE 
program using the SRA Reading and Language programs. Teacher demo- 
graphics are listed in Table 1-14. The costs are detailed in the cost report. 
There is an additional stipend for certified bilingual or ESL teachers. 



Materials and Assessments 



Curriculum 

Curriculum is the SRA Reading Mastery and the SRA Language for 
Learning. These are refined versions of the curriculum materials used in 
Project Follow Through. The SRA curriculum is supplemented with trade 
books, i.e., children’s literature. However, unlike these earlier programs that 
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typically began in first grade, RITE begins in Pre-K. 

Assessments 

Formative. Students’ reading progress is assessed every five lessons using 
Mastery Tests that accompany the program. Students complete a mastery 
test on phonemic awareness and phonics awareness and other skills taught. 
Also, reading fluency rate and accuracy is recorded. In Level 2, only read- 
ing fluency is assessed. For the language program, the curriculum progress 
rate of each instructional group is reviewed to assess whether the group is 
progressing at an acceptable rate. It is important to note that student 
growth in reading fluency is carefully reviewed by the trainers and the prin- 
cipals, and that classrooms are visited at least once a week. Data are exam- 
ined to see which students need extra help and/or need to be placed in a 
more accelerated or less accelerated instructional group. 

Summative. The Stanford Achievement Test Series, Ninth Edition (S AT- 
9) is given each spring beginning in kindergarten. This test has reading, 
pre-reading, and language sections. Some schools test students in both fall 
and spring on the Iowa Test of Basic Skills. Student growth on these meas- 
ures is carefully reviewed. 

Progression of Students through the Program 

Best case scenario of a progression. Students will complete SRA Reading 
Mastery Level II by the middle of second grade and then enter either 
Horizon (a basal reader based on principles of direct instruction) or a typ- 
ical district basal series such as HBJ or Open Court. 

Worst case scenario of a progression. A student has not completed 
Reading Mastery-One by the end of first grade. If this is the case, the 
supervisor/trainer works with the teacher to assess what has gone wrong. If 
problems persist, the student is moved to a classroom where the teacher is 
deemed to be more proficient. 

Language Proficiency Tests 

For Pre-K, Kindergarten, and first grade, only oral language proficiency is 
assessed. The measure used is the Language Assessment Scales. This is the 
most commonly used oral language measure in the US. Houston Indepen- 
dent School district follows conventional criteria: Students with scores of 1, 
2, or 3 are considered LEP. Those with higher scores are no longer consid- 
ered LEP. 

In second grade and beyond, an assessment of written proficiency in the 
English language is also required for both classification as LEP and ulti- 
mately for declassification. In grade 2 and grade 3, scores on the Language 
subtest of any standardized achievement test such as the SAT9, ITBS, or 
CAT are used. To lose the LEP classification, students must score at or 
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above the 40th percentile on these nationally standardized tests on the Lan- 
guage subtests. Beginning with the end of third grade, scores on the Lan- 
guage sub test of the TAAS (Texas's state assessment) are used for classifi- 
cation decisions. 

Staffing Model/Teacher Training/Professional 
Development 

The RITE program involves extensive professional development. Note 
however, that the professional development is no more or less extensive for 
teachers with English-language learners in their classes than for those with 
native English speakers. 

Staffing Model 

There are typically 22 students per teacher. Most teachers have a parapro- 
fessional assist for about one hour per day. Some of the Tide One schools 
have CRSTs (class reduction size teachers). In those rooms, the ratio for 
reading and language arts is 16:1. Again, the classroom reduction teacher 
program is not unique for classes with English-language learners. It is part 
of a district initiative to improve achievement by lowering class size for 
reading and language arts. 

Certification 

Most of the teachers who work with English-language learners are certified 
bilingual education teachers or have an ESL endorsement. Some are work- 
ing on certification in HISD’s alternate certification program. Only a very 
small portion are not certified at the current time. 

Professional Development, Training and Monitoring 
of Reading and Language Arts Instruction 

Regularly scheduled trainings 

Professional development is extensive in these schools. It begins with a 
three-day summer institute. There is a foUow-up Saturday institute in 
October. Additionally, there are monthly two -hour training sessions held 
throughout the year. 

During the formal training sessions, teachers learn the direct instruction 
teaching techniques, the subtleties of instructional strategies such as error 
correction, and how to maintain a high success rate during a lesson 

They also learn about program extensions for English-language learners 
such as how to build vocabulary and test taking skills. They also hone in on 
how to teach key phonemic skills such as sound blending. Content for these 
meetings may evolve from problems the trainers see in their weekly class- 
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room observations, or issues raised by teachers in their debriefing vvith the 
trainers. 

In addition, each classroom is observed at least once a week by the train- 
er. The trainer also reviews the weekly assessment data. During brief infor- 
mal meetings, teachers receive ongoing technical assistance on proper use 
of the programs, how to use assessment information to provide additional 
practice to students struggling with a new skill or concept, and how to tai- 
lor instruction to meet the needs of English-language learners with the 
direct instruction programs. 

A major focus in the informal weekly meetings with the trainer is review 
of the student academic progress data and how to use these data to make 
instructional decisions. Teachers also receive feedback on weekly observa- 
tions from the trainer. The trainer may work with the teacher on specific 
teaching techniques or strategies that seemed problematic, or answer any 
questions teachers have. 

Sheltered English techniques are taught to all teachers in the RITE pro- 
gram, even those who currendy have no English-language learners in their 
class. 

Paraprofessionals also receive training. The most skilled* actually teach 
reading to small groups. Others grade papers, and monitor students doing 
independent work and answer questions students may have while working 
independendy. The assistants are in the room for only one hour per day. 
The paraprofessionals receive the same essential training as the teachers ex- 
cept it is adapted to what their classroom tasks actually are. 

Incentives/Recruitment 

Graduate credit is given for training with the University of Houston. Thus 
teachers receive graduate credit for their participation in the RITE pro- 
gram. Teachers with certification in bilingual education or ESL receive an 
additional stipend of approximately two to three thousand dollars. This is 
district policy. 

There are also several other incentives. Each year there is a special din- 
ner at the Astrodome for RITE teachers. Small gifts such as sweatshirts are 
given as morale boosters. 

Compensatory Instruction 

Summer school is available for English-language learners as well as native 
English speakers. Between 50 and 70 percent of students attend the four- 
week summer school program. Another special feature for all Tide One 
schools, including the RITE schools, is the CSRT program. Effectively this 
reduces class size for reading/language arts from 22 to about 16. A special 
teacher pulls out six students for their core reading and language arts 
instruction in many rooms. Thus the teacher is only responsible for teach- 
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ing 16. This is not unique for English-language learners. 
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Seattle School District s 
Elementary Schools 

The Bilingual Orientation Center Programs 

Scott K. Baker 
Eugene Research Institute 

Seattle School District Bilingual Orientation Programs 
for English-Language Learners 

Programs for English-language learners in the Seattle School District have 
not changed substantially since 1996 when the READ journal reported on 
the district’s success \vith this population (Baker, 1996). The state of 
Washington mandates bilingual education for students with a primary lan- 
guage other than English, but many “bilingual” programs, such as those in 
Seattle, do not provide extensive primary language instruction. They are 
called bilingual programs, but teach primarily in English, using structured 
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immersion techniques. Students may be supported in their primary lan- 
guage during instruction by an instructional assistant, but the dominant 
language of instruction is English. 

Washington state law requires transitional bilingual education programs 
when there are sufficient numbers of LEP students from the same language 
background, or, where instruction in two languages is not practicable, as 
established by the district superintendent, the state law allows an alterna- 
tive system such as ESL to be used. 

In September 1993, Evergreen Legal Services, a public advocacy organ- 
ization, brought suit against the Seattle School District on behalf of a stu- 
dent, Sang Van, and the class of all limited- English proficient students and 
their guardians (Sang Van v. Seattle School District), A number of complaints 
were alleged in the lawsuit, including a lack of certified native language 
teachers, inappropriate assignments of LEP students, locations of bilingual 
program schools, and lack of sufficient materials in the native language. 

Seattle's LEP population had grown rapidly in the 1980’s to 6,000 stu- 
dents (out of a total enrollment of 45,000 in the district) fiom 90 different 
language backgrounds. While the district tried different variations of bilin- 
gual and ESL programs, by the 1990’s they determined that it was no 
longer feasible to implement full bilingual instruction programs for every 
language group. Thus, the district availed itself of the provision in state law 
for providing alternative programs. This policy on LEP students coincided 
with the implementation of a new School Based Management initiative in 
the Seattle schools that allowed each school to design its own organization 
and provided funding to support smaller classroom size, team teaching, and 
other features. 

On April 18, 1995, a settlement was signed by all parties to this suit 
which set forth in detail the way in which the district meets its obligations 
to LEP students, with regard to monitoring academic achievement, cur- 
riculum development, staff recruitment and training, and other program 
features. This settlement did not force the Seattle district to change its pol- 
icy on the education of LEP students, and the district was not directed to 
provide more native language instruction than it deemed suitable. Due in 
large part to the political pressures brought on by the lawsuit, the district 
refers to its programs for LEP students as “bilingual” although they do not 
meet the definition of such a program in their implementation and actual- 
ly closely resemble English immersion. 

The primary program for English-language learners who are new arrivals 
to the district is the Bilingual Orientation Centers, In this report, I first pro- 
vide historical background on the growth of the English-language learner 
population in Seattle’s schools. Then, I discuss the Bilingual Orientation 
Centers, focusing primarily on programs in the elementary schools, but also 
touching on the one secondary program. In the last part of the report, I 
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briefly describe the regular program English-language learners typically 
transition into when they exit the one of Bilingual Orientation Centers. 

Background 

In the fall of 1970, the Seattle School District established an official pro- 
gram for English-language learners. At that time, the program served only 
about 100 students. By 1993, 11,117 of the district’s 44,962 students (24.7 
percent) had a non-English language spoken at home. Of these 11,117 stu- 
dents, 6,185 (55.6 percent) were in programs for LEP students. 

Before 1970, students were informally served at individual schools 
throughout the district. The end of the Vietnam War resulted in a tremen- 
dous increase in the number of students from Southeast Asia who entered 
Seattle schools. This increase continued until approximately 1982, when a 
shift in the student population occurred. 

The influx of large numbers of English-language learners increased, but 
these new arrivals represented many geographic regions of the world 
including large sections of Asia and Latin America. Since 1996, there has 
been yet another change; new arrivals have come increasingly from 
Spanish-speaking countries. Currently, there are over 100 major native lan- 
guages spoken at home by students and their families in the Seattle School 
District. 

Students are served in the district’s programs for English-language 
learners if their native language is not English, and they earn a language 
proficiency score of 1, 2, or 3 on the oral sections of the Language 
Assessment Scales (LAS) (De Avila & Duncan, 1987). Exit criteria, for 
determining that students no longer need services, is based on a scholastic 
achievement test to determine that students are adequately prepared for fuU 
time English language instruction in the mainstream. Students are typical- 
ly exited from programs if they score at or above the 35th percentile on a 
standardized test of reading and language proficiency. 

The Seattle School District also exits students from programs if district 
personnel believe students have the ability to perform successfiiUy in the 
mainstream, and parents choose to discontinue service, which they indicate 
by signing a waiver, even if the student has not scored at the 35th percentile 
on a standardized test of reading and language. In these cases, teachers with 
expertise in working with English-language learners review the student’s 
academic progress reports, and discuss with the general education teacher 
the possibility of a successful mainstreaming experience for the target stu- 
dent. 

After such a review, a decision to “trial exit” a student may be made. The 
ESL teacher and classroom teachers review a student’s classroom progress 
at each marking period. If the student is making adequate progress, the 
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child is no longer directly served. A student may not remain on trial exit 
more than one academic year. During the reintegration, the student may 
still receive some native language support from a bilingual instructional 
assistant on an as-needed basis. 

Bilingual Orientation Centers: 

Characteristics of the Students Served 

The well publicized success of the regular programs for English-language 
learners in the Seattle School District depends, in part, on the quality of the 
Bilingual Orientation Centers that prepare many students for these pro- 
grams. Students who have recendy arrived in the United States, or are new 
to the Seattie School District, and have very limited English proficiency 
usually attend one of five Bilingual Orientation Centers to help prepare 
them for the regular programs for English-language learners. Students typ- 
ically stay in one of the centers at least until the end of the semester in 
which they arrive. Many students remain until the end of the next full 
semester if this will better prepare them for the regular programs. 

The primary purpose of the Bilingual Orientation Centers is to prepare 
students as quickly as possible for successful experiences in the regular 
bilingual program for English-language learners. There are four elementary 
Bilingual Orientation Centers in the district. 

The percentage of students by native language group is presented in the 
following table. Each program has approximately 60 students at any given 
time during the year. Because the number of students entering the pro- 
grams and exiting into the regular programs for English-language learners 
changes dramatically during the course of the school year, the number of 
children served in the centers is much higher. The table shows that the 
major language group being served in each center is Spanish. This repre- 
sents a substantial change from five years ago when the predominant lan- 
guage group served was Vietnamese. 
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Table T 16 . 

Seattle Schools 

Elementary Bilingual Orientation Center Language Populations 

December 2000 





T. Marshall 


John Stanford 


TOPS 


Fairmont Park* 


Spanish 


55% 


67% 


49% 


100% 


Somali 


11% 


0% 


12% 


0% • 


Mixed Chinese 


6% 


7% 


8% 


0% 


Oromo, Amharic, Tigrigna 


9% 


0% 


0% 


0% 


Japanese 


0% 


10% 


8% 


0% 


Filipino languages 


11% 


4% 


0% 


0% 


Viemamese 


5% 


6% 


16% 


0% 


French 


0% 


0% 


6% 


0% 


Other 


3% 


6% 


7% 


0% 


*Start-up program 



Entry/Exit Criteria 

All English-language learners who are new arrivals to the country are 
assessed on the LAS to determine eligibility for one of the Bilingual 
Orientation Centers. If students score less that 50 percent on the oral sub- 
tests of the LAS — lower than approximately the 33rd percentile — they are 
eligible for the program. 

Eligible students can either participate in one of the Bilingual Orienta- 
tion Centers or attend their neighborhood school and receive ongoing 
tutoring from an instructional assistant, who frequendy will be fluent in the 
student’s native language. The majority of parents opt for the district’s rec- 
ommendation, which is to first attend a Bilingual Orientation Center, and 
then transition into the regular program for English-language learners. 
Approximately one-quarter of the parents choose to bypass the Bilingual 
Orientation Center, typically so they can send their child direcdy to the 
neighborhood school, where they participate in the regular program for 
English-language learners. 

Exit criteria from an elementary BOC does not include performance on 
the LAS. Instead it depends on consistent student performance on a dis- 
trict BOC English-language learner Process Report detailing how well stu- 
dents are doing in their ability to (a) comprehend and speak English, (b) 
read in English, and (c) write in English, Students are regularly evaluated 
on their emerging abilities in these areas and when they demonstrate con- 
sistent use (as opposed to developing or needs more time)y they are moved 
toward program exit. For elementary students, consistent use in approxi- 
mately seven descriptive categories signals the teaching staff that the stu- 
dent may be ready for the regular program. When students transition to the 
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regular program for English-language learners, they are monitored closely. 
The typical scenario is that they receive a more concentrated level of serv- 
ice from an ESL teacher, and an instructional assistant who is assigned to 
work with them for the first few months, to make sure the student is under- 
standing the lessons and is able to keep up with the assignments. 

Curriculum Materials 

Preparation in a Bilingual Orientation Center involves two broad strands. 
First, students are provided with intensive ESL instruction. For those stu- 
dents whose native language is one that the district has trained instruction- 
al assistants in, primary language support is provided on an as-needed basis 
to support English language acquisition and to allow students to continue 
to make progress in general content knowledge while they are learning 
English. Second, focus is placed on helping students who may be unfamil- 
iar with schooling in the U.S. generally, or with the Seattle schools specifi- 
cally, navigate the public school system. Like other urban districts, many 
recent immigrants to Seattle have attended school sporadically in their 
native countries. These students benefit from opportunities to learn and 
experience broad aspects of how schools function -in the United States. 
Bilingual Orientation Centers help students learn and experience what 
schools are like in this country in a non- threatening atmosphere. 

The instructional approach in the Bilingual Orientation Center class- 
rooms is based on a natural approach to language development. The ratio is 
ideally not more than 20 students to 1 certified teacher, and an instruction- 
al assistant is usually assigned to each classroom for a large part of the day. 
Staffing adjustments occur, involving certified personnel, as the number of 
students in a center change during the year. Focus is on the rapid develop- 
ment of communication skills through many interactive instructional 
strategies, including oral modeling with sentence frames, questioning 
allowing non-verbal responses, language experience, and activities aimed at 
kinesthetic learners. There is a consistent focus throughout the centers on 
achieving very high levels of student use of the English language in the 
classroom through a wide array of activities. 

The curriculum is best described as eclectic, with an emphasis on 
immersing students in authentic and realistic learning situations. The fol- 
lowing is a list of specific materials and activities used frequently in 
Bilingual Orientation Center classrooms. 

l.The Science Research Associates (SRA) Photo Library contains thou- 
sands of realistic pictures and suggested activities that teachers use for a 
variety of purposes. The photos are used to initiate and organize discus- 
sions, to sort things by categories, as a simple labeling activity, and to 
engage in more complicated verbal interactions such as sequencing. 
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2. The Picture Dictionary by Hampton Brown is used to stimulate inter- 
actions involving more academic types of language, while maintaining a 
focus on authenticity. A similar product by Oxford, from the National 
Textbook Company, has also been popular. The best of these materials also 
contain transparencies that teachers and students can use to engage in more 
extended discussions. 

3. Nystrom Maps and Globes and Rand MacNally Bigbooks are among 
the most popular materials teachers use for introducing students to geogra- 
phy and geographical concepts. 

An important point, and frequent problem, has been accessing curricula 
materials that are large enough so that all students can see them during a 
teacher-led lesson, and facilitate student observation of important features 
and details. It is problematic that whereas large visual materials are readily 
available and appropriate for younger students this is not the case with older 
students. 

Teachers are increasing their use of projectors with the capacity to dis- 
play material from CD-ROMs. CD-ROMs have enormous potential to 
provide the types of visual image teachers say their students need, and there 
are many sources appropriate for older students. One of the most popular 
has been the CD-ROM interactive material from Steck-Vaugbn. Others 
include: 

■ National Geographic materials — ^in particular their videos — are very pop- 
ular with teachers. The children’s magazine is too difficult for students to 
read, but the pictures and concepts can be used as a source for discus- 
sions. Zoo Books from National Geographic are popular because they deal 
with a favorite subject (animals) and do so one animal at a time. 

■ Easy Aides materials are used to build extended communication skills. 
The 8.5 by 11 inch line drawings, such as “Tell the Doctor,” are used to 
initiate discussions about common every-day themes. 

■ Scholastic materials, especially Bigbooks, are a popular in teaching read- 
ing. Adaptations of common fairy tales contain lots of rhymes and repet- 
itive words that are appropriate for older students. Beginning reading is 
a real strength of the program. The very specific and goal oriented learn- 
ing context helps students focus, and the face-to-face interactions pro- 
mote numerous opportunities for verbal interactions. 

■ Phonics instruction is a critical component of reading instruction, help- 
ing students link sound-symbol relationships to meaningful words in a 
new language. 

The Bilingual Orientation Centers use a combination of language expe- 
rience and phonics in their instruction. Sight word reading is popular 
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because students can match words to pictures and develop vocabulary and 
reading simultaneously. 

Phonics is used to provide students with the foundational skills they need 
to read on their own. The degree of whole word reading to phonics varies 
among learners and teachers. But there is a great deal of consistency among 
teachers in stressing the importance of contextualized learning. 

Instruction is theme based. Teachers share the idea that if students can 
begin to master certain terms and concepts in a narrowly defined area (i.e., 
a specific theme) they can quickly become an “expert” in an area and engage 
in meaningful discussions and extended, challenging learning opportuni- 
ties. This type of learning situation is also conducive to working on more 
formal aspects of language development, such as changing tenses and syn- 
tax. 

Extensive review is built into the lessons and curriculum. During each 
week, approximately one-half of a day will be devoted explicidy to review- 
ing previously taught material. This is in addition to the extensive review 
teachers do every day on an ongoing basis during instruction. Some of this 
instructional review is done in small groups, where students take more and 
more responsibility for directing the learning situation. Games, such as 
Content Bingo, are also used as a way to review. 

Finally, field trips are used extensively. They provide the context for 
learning opportunities that can last a week or longer. There is a minimum 
of four field trips per year, but many of the center classrooms exceed this 
number. 



Progress of Students through the Program 

Teachers gauge student progress on an ongoing basis through daily class- 
room interactions. The goal is improved proficiency with academic and 
communication skills in English. Formal progress reports are used to mon- 
itor how well students are moving toward exiting the program. Students 
typically are ready to exit the program after they have been in the program 
for at least one semester. Only occasionally do students stay in the program 
longer than one year. However, the number of students who are staying 
longer than one year is increasing. The district attributes this to a growing 
percentage of students arriving from very high poverty backgrounds, and 
most importantly to the growing percentage of students arriving who have 
sporadic literacy experiences in their home countries. 

Staffing Model/Teacher Training/ 

Professional Development 

All new teacher hires in the Bilingual Orientation Centers have a bilingual 
endorsement from the state of Washington (some have been grandfathered 
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into the system). The endorsement allows them to teach in one of the 
Bilingual Orientation Centers or in the regular program for English-lan- 
guage learners. In many of the centers, the teachers have been teaching in 
a center classroom for many years, sometimes 20 or more. A teaching posi- 
tion in one of the centers is considered an excellent assignment. Students 
are highly motivated, they work hard to learn the content and the new lan- 
guage, there are few behavior problems, and the curriculum is enjoyable for 
teachers. 

Teachers have a great deal of flexibility in developing lessons, and the 
goal of concentrating on realistic and authentic learning situations to build 
students’ communication skills is one that most teachers find interesting 
and challenging. 

It is considered an advantage that the Bilingual Orientation Centers 
essentially operate as a school-within-a-school. Center classrooms are not 
as closely tied to the district’s standards for academic outcomes as other 
programs because the focus and goals are different. The district is trying to 
align the centers more closely to district standards, but they are attempting 
to do so in a way that makes sense for students who are very new at learn- 
ing English. 

Staffing Model 

At the beginning of the school year the staffing ratio in each of the Biling- 
ual Orientation Center classrooms is approximately 14 to 1. It increases 
after that and usually stays close to the 20 to 1 limit. If it gets higher than 
20 to 1, every effort is made to hire an additional certified teacher. 

Instructional assistants play a key role in the Bilingual Orientation Cent- 
ers, Whenever possible, assistants speak the native language of students in 
the center and provide support in the student’s native language during 
instruction. However, there are close to 100 different native languages spo- 
ken in the district, and native language support is provided in only a small 
percentage of those languages. In terms of the actual numbers of students 
who receive native language support, the percentage is quite high because 
most of the students speak a language the district is able to support. 

Each Bilingual Orientation Center has the equivalent of two full-time 
instructional assistants, and most centers budget for more than the mini- 
mum required. Each building must fund one full-time instructional assis- 
tant through building funds; the rest are funded by central administration. 
Instructional assistants must have attended two years of college either in the 
U.S. or another country. Their English proficiency is also judged during an 
interview and must be considered acceptable. At the secondary center, the 
instructional assistant must also be proficient in mathematics and be able to 
assist students in that subject area. 

Professional development for instructional assistants begins with a half- 
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day orientation. Additionally, most buildings provide ongoing training and 
support as part of the building plan. At schools with more experienced 
teachers, there is more support for the ongoing training of instructional 
assistants because experienced teachers believe that having highly trained 
instructional assistants, improves program quality significantly. 

Ongoing professional development is not connected to the salary struc- 
ture for instructional assistants, but they are paid to attend any training 
activities that occur outside of normal work hours. 

Professional Development and Ongoing Training 

Professional development for teachers, and instructional assistants, in the 
Bilingual Orientation Centers is provided through a number of contexts. 
Budgeting in the district is primarily building based, which means the prin- 
cipal and staff have a great deal of leverage in determining budget decisions 
for the school. One of the best avenues for professional development comes 
from teachers serving as their own advocates, i.e., making a case that they 
or their instructional assistants need certain training and workshops to bet- 
ter serve their students. 

The district also plays a key role in providing professional development 
opportunities to teachers in the Bilingual Orientation Centers. Teachers are 
typically provided support to attend two professional development work- 
shops per year. The district also provides support for some teachers or 
administrators to attend professional conferences. 

Professional development is not a requirement in the district, but nearly 
all of the teachers engage in some type of ongoing professional develop- 
ment, partly because these activities are tied to the district's salary structure. 

There is also an annual summer institute put on by the Department of 
Teaching and Learning in the district. This is open to both certified bilin- 
gual education teachers and general education teachers as a way to foster 
beneficial collaboration and to promote more seamless services to English- 
language learners. The major benefit to the general education teachers is 
that they learn ways to make content more accessible to English-language 
learners through verbal interactions between students and teachers, and 
among students working together. 

The Bilingual Education Department in the district also writes a 
bimonthly newsletter, A portion of the newsletter offers “tips for teachers” 
in working with English-language learners. 

Incentives/Recruitment 

Recruitment is not as difficult for teachers at the Bilingual Orientation 
Centers as it is for other programs in the district. Usually word of mouth is 
enough to keep positions filled with strong teachers and support staff. The 
program is viewed very positively among district personnel, which helps 
draw qualified and skilled teachers. Also, television coverage in the local 
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area has been positive and has resulted in high interest among teachers to 
work in one of the Bilingual Orientation Centers, 

There is a full-time teacher-consultant, Nancy Burke, who serves the 
program as an administrator. Ms. Burke taught in the district's programs 
for English-language learners for many years and her contribution to the 
ongoing success of the program is critical. She regularly attends profession- 
al conferences (e.g., TESOL) and recruits teachers to the district. 

Ms. Burke also spends considerable amounts of time troubleshooting in 
the Bilingual Orientation Centers. Her job, in the context of the Bilingual 
Orientation Centers, is to make sure the centers are effectively teaching 
students what they need to know to be successful in the regular program. 
And because Ms, Burke has expertise in second language learning and 
instruction, her judgments regarding program quality are critical in main- 
taining strong programs. 

Because students stay in the Bilingual Orientation Centers such a short 
amount of time, there is a natural consequence when the center is not doing 
its job as well as it should. The result is that students are not being prepared 
appropriately, which results in their having a more difficult time than they 
should comprehending, and when they are transitioned. 

The state also is taking an increasing interest in how long students are 
being instructed in programs for English-language learners. This is true for 
the Bilingual Orientation Center programs and the regular programs for 
English-language learners. Increasing pressure is being put on schools to 
demonstrate that they are actively preparing students for general education 
classrooms, and to take the state- mandated achievement tests and do well 
on them. 

Seattle’s Regular Program for English-Language Learners 

Programs for English-language learners have two distinct phases. First stu- 
dents enter a Bilingual Orientation Center. These centers provide intense 
English language development for approximately one year to new arrivals 
in the district to prepare students for the district’s regular program for 
English-language learners. 

Once students complete their work at the Bilingual Orientation Center, 
they enter the regular program for English-language learners. The major 
feature of this program is ESL instruction. ESL classes are designed to help 
students improve their listening, speaking, reading, and writing skills in 
English. At the secondary level, sheltered English instruction, with a small 
amount of native language support furnished by bilingual instmctional 
assistants, focuses on content acquisition in math, social studies, science, 
and health. Aside from these ESL classes, English-language learners 
remain in the , general education classroom throughout the day. 
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Instructional assistants may be assigned to these classes to provide primary 
language support. 

At the elementary level, traditional ESL pullout models have given way 
to more integrated approaches in which eligible students are served by cer- 
tified staff in the general education classroom. Many schools in the Seattle 
School District use a combination of models that allows those students who 
have more limited skills or who have missed significant years of schooling 
to receive additional support in a more traditional pullout setting. 

There are five primary models in the Seattle School District program for 
English-language learners. Schools are encouraged to be flexible in how 
they use these models or combine them in ways that make the most sense 
given the needs of their students. These models are briefly described. 

Pull-Out Model. Students are assigned to a mainstream classroom and 
are “pulled-out” to the ESL classroom for service. These pull-out times vary 
according to the needs of the students and scheduling issues in the school. 
Overall, these pull-out classes last 30-60 minutes and serve groups ranging 
from 5-12 students. 

Pull-In Model. The staff trained to work with English-language learn- 
ers is “puUed-in” to the mainstream classroom to provide services. This fre- 
quently includes a bilingual Instructional Assistant who has been trained by 
the certified teacher and is fluent in the student’s native language. Certified 
teachers trained to work with English-language learners also frequently 
provide services in the general education classroom. This integrated service 
provides a good opportunity for English-language learners to learn English 
that is most directly relevant to the content of the general education class- 
room. 

Basic Skills Block. Students are pulled into a basic skills block on a 
schoolwide basis. Besides ESL instruction, basic reading instruction is pro- 
vided by the teacher certified to work with English-language learners. In 
this model, service is typically delivered for approximately two hours per 
day. Class sizes average between 15 and 24 students. Many native English- 
speaking students also participate in the basic skills blocL 

Blended Model. Students are served in “blended” groups which may 
include students officially in programs for English-language learners, spe- 
cial education, Title I, or served only in general education. Various combi- 
nations of staffing patterns may be used to deliver instruction, depending 
on the content and the students in the class. Typically, however, blended 
classrooms are team-taught and the class sizes are reduced. 

Tutorial Model. Trained tutors, who are usually fluent in the student’s 
native language, tutor English-language learners on an hourly basis. 
English-language learners who choose not to attend one of the traditional 
center schools are provided with tutors. In the typical tutoring model, stu- 
dents remain in their general education classrooms and tutors come on a 
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part-time basis to work with them. Each LEP student receives direct tuto- 
rial service for up to three hours per week. 

Data on the Effectiveness of the Seattle Program 
for English-Language Learners 

1. Seattle’s implementation of an English immersion program, with some 
native language support, resulted in impressive achievement outcomes for 
the district’s 6,000 students in special programs for English-language learn- 
ers. 

2. Students enrolled in programs for English-language learners made 
significant achievement gains compared to the district average, and gains 
appear to be direcdy attributable to participation in these programs. Stu- 
dents make the most rapid gains when they participate in these programs 
for up to approximately four years. When students remain in programs 
longer than four years, the gains seem to dissipate considerably. 

3. Students who successfully exit from special programs earned achieve- 
ment scores at rates that are comparable to district averages. This finding is 
true for all major subject areas, including reading, language, and mathe- 
matics. 

4. The dropout rate for students in special programs is considerably lower 
than district averages. 

5. The graduation rate for students in programs for English-language 
learners was considerably lower than the district average. For students who 
successfully exited from programs for English-language learners, high 
school graduation rates are on par with other students in the district. 

6. The overall perception of students, parents, and staff is that the special 
programs have a positive influence on the educational experiences of 
English-language learners. (See Baker [1996] for further details.) 
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PhaselV 

Incremental Costs 
AND Resources Available 
TO Fund English Acquisition 

Programs 

IN THE Nogales Unified 
School District 



Executive Summary 

I n February 2001, the Arizona Department of Education (ADE) con- 
tracted with two entities for a cost study of English Acquisition pro- 
grams in the Nogales Unified School District. The Research in 
English Acquisition and Development Institute (READ Institute) 
was tasked to provide a qualitative description of the students served by 
each program currently used in the district to fulfill Lau v. Nichols {Lau) 
requirements for limited English proficient (LEP) students. Additionally, 
the READ Institute was asked to identify the program elements that 
address unique or extraordinary circumstances of the LEP population in the 
district. The ADE also contracted with Sjoberg Evashenk Consulting, 
LLC, to identify and analyze the costs related to the program elements that 
address the unique needs of Nogales’ LEP population that cannot be met 
within a Structured or Sheltered Immersion program, and to project the 
resulting costs into 2002 Arizona cost of living dollars. Furthermore, 
Sjoberg Evashenk Consulting, LLC, was asked to identify all funding 
sources used by the district to fund its current English Acquisition pro- 



COST OF ENGLISH ACQUISITION PROGRAMS 

65 





grams and project funding available in fiscal year 2002 dollars in total and 
per pupil. 

This report conveys the results of Phase IV of the four-part cost study 
where the two firms identified the program elements and cost components 
that address the unique needs of the LEP population in Nogales as well as 
the funding related to providing LEP services. Generally, both Sjoberg 
Evashenk Consulting and the READ Institute found that the Nogales dis- 
trict is in an isolated area of southern Arizona on the Mexican border. The 
community interacts heavily with its adjacent Mexican “sister cit/’ and 
Spanish appears to be the primary language spoken on both sides of the 
border. The district, however, believes it essential for students to attain lit- 
eracy in English and provides, for the most part, Sheltered or Structured 
Immersion programs for its LEP students. LEP students are “main- 
streamed” and provided with language arts or English as a Second 
Language (ESL) classes to improve their English skills. Those with very 
limited English skills are provided extra services to bolster learning with the 
goal of quickly moving these students into mainstream classes. Further, the 
Nogales district also offers “Bilingual Education Programs” intended to 
attain literacy in Spanish as a “second language” course. 

The READ Institutes review found that, in the Nogales community, 
Spanish is used more regularly than English and it is possible to pursue a 
life in the area only speaking Spanish. According to the READ Institute, 
this accounts for the large number of LEP students enrolled in the Nogales 
district schools. These students have not been exposed to English and, 
therefore, are not prepared for academic learning in English. Nogales’ 
unique environment requires that the schools provide special help in second 
language learning from the first day of enrollment. Each school within the 
district offers specialized program support to assist their LEP students; for 
instance, the elementary schools offer a combination of immersion, early- 
exit transition, and late-exit bilingual instruction while the middle schools 
and Nogales High School offer ESL programs. Further, not only does the 
Santa Cruz Alternative High School offer support in Spanish only when 
necessary, but also each school enriches students’ academic experience by 
providing Spanish language classes, which help maintain the viability of the 
community language and culture. Generally, achievement scores through- 
out the Nogales schools tend to be in the low-average to average range. 

Sjoberg Evashenk Consulting, LLC, was assigned with identifying the 
cost elements related to providing services that address the unique needs of 
the LEP population in Nogales. We found that, similar to other Arizona 
school districts we contacted, the Nogales district also incurs some addi- 
tional or incremental costs associated with providing services to LEP stu- 
dents. These incremental components mainly consist of supplementary 
materials, additional teaching staff, and adult and parent education pro- 
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grams, as well as bonuses paid to English as a Second Language (ESL) or 
bilingual endorsed teachers and supplemental staff training in LEP teach- 
ing methods. Overall, the district incurs incremental costs of roughly $1.3 
million, or $331.60 per pupil, to provide services to its LEP students in the 
current school year. The district’s elementary schools incur $677,317 of the 
total incremental costs, which is more than either the middle or high 
schools, which incur incremental costs in the amounts of $357,342 and 
$268,874, respectively. Projected to 2002, total incremental costs grow to 
$1,329,604. 

However, the current design of Nogales’ Structured or Sheltered 
Immersion programs may not fully address the unique and extraordinary 
needs of its LEP population. Specifically, because of the prevalent use of 
Spanish throughout the community, in local television and radio, in busi- 
nesses, on school playgrounds and in hallways, students have very litde 
exposure to the English language outside the classroom environment. 
Although Nogales immerses most of its LEP students in mainstream class- 
es taught in English, the students in Nogales do not receive sufficient rein- 
forcement of their English skills outside the classroom and that, may add to 
the difficulty in learning the language. Further, to function successfully in 
the Nogales area, a student faces the unique need to be literate in both 
Spanish and English. 

To address these distinct circumstances, Sjoberg Evashenk found that 
the Nogales district is implementing several initiatives within its sphere of 
control. For instance, efforts have been made to recruit additional bilingual 
staff, make LEP programs coordinated and consistent across schools, and 
improve student performance on test scores. In addition, the district vwH 
continue to offer English education to adults and parents in Nogales, help- 
ing the English language more fully penetrate into the community and 
homes where students spend their time — thus creating an environment 
where students have more exposure to English. While Sjoberg Evashenk 
believes these efforts will assist in addressing the unique needs of the com- 
munity and cost the district $277,600, it appears unlikely that these initia- 
tives are sufficient to mitigate the lack of daily external exposure to the 
English language necessary to support viddespread English proficiency in 
Nogales’ student population. 

To fund its LEP programs and initiatives to address the unique LEP stu- 
dent needs, Nogales reported more than $1.8 million of total funding avail- 
able, or $365.86 per LEP student, for the 1999-2000 school year. This 
amount has increased to more than $1,876,867, or $477.45 per student, for 
the current school year. However, Sjoberg Evashenk’s projections indicate 
that LEP funds available for the 2001-2002 year will decrease to 
$1,747,245, or $411.56 per student, primarily due to a reduction in LEP 
student enrollment. It is important to note that these per student calcula- 



COSr OF ENGLISH ACQUISITION PROGRAMS 

67 




tions may change based on revisions currently being calculated by the 
Nogales district; however, as of May 1, 2001, the district had not yet pro- 
vided us their revisions. Further, while the calculations of funding available 
and incremental amounts suggest a “gap,” these amounts are not directly 
related or comparable. Sjoberg Evashenlds review reveals that the total 
sources of funding, as reported to the ADE by the Nogales district, related 
to LEP programs have no relationship to the level or amount of incremen- 
tal costs incurred by the district. 

Chapter 1 

Nogales Unified School 
District Programs 
FOR Engush-Language 
Learner Students, K-12 



Rosalie Pedalino Porter 
Scott K. Baker 

Chapter Summary 

The Arizona Department of Education defined the tasks in Phase 4 of the 
English Acquisition Program Cost Study, as related to program descrip- 
tions for the Nogales Unified School District, in these two broad cate- 
gories: 

A. Define the English Acquisition Programs currently used to fulfill Lau 
requirements for Limited English Proficient students in the Nogales 
Unified School District in Nogales, Arizona. Include a brief qualitative 
description of the students served by each program. 

B. Identify the elements of each program currently used to fulfill Lau 
requirements that address unique or extraordinary circumstances of the 
Limited English Proficient students in the Nogales Unified School 
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District. These should only include elements necessary to meet students’ 
needs which cannot be met through a Structured or Sheltered English 
Immersion program. 

These parts of Phase IV of the cost study are the responsibility of the 
Institute for Research in English Acquisition and Development (READ) 
and are reported here, while parts C, D, and E have been investigated and 
are reported by Sjoberg Evashenk Consulting. Parts A through E of the 
Phase IV report are combined and submitted jointly by the two organiza- 
tions. 

Background of the Cost Study: 

While the authors of this study are not attorneys, it is useful to provide a 
brief review of the legal background, in order to understand better the exist- 
ing conditions in Nogales and in Arizona leading to the Cost Study. 
Available documents are reviewed, as follows: 

1. Judge Marquez’s Order for a Cost Study 

On January 24, 2000, Judge Alfredo C. Marquez, Senior U. S' District 
Judge in the United States District Court, District of Arizona, signed an 
Order for a Cost Study in the matter of Miriam Flores et al. Plaintiffs v. 
State of Arizona, et al. Defendants, 

The aforementioned Flores v. Arizona court suit was initiated in 1992 
and alleged that the State “was failing to provide limited-English proficient 
(LEP) children with a program of instruction calculated to make them pro- 
ficient in speaking, understanding, reading, and writing English, while 
enabling them to master the standard academic curriculum as required of 
all students.” Judge Marquez’s Order describes in detail the repeated delays 
by the State of Arizona in implementing the Cost Study until the present 
time. There is serious concern expressed in this document about the insuf- 
ficient level of funding from the State with which local districts are to cover 
the extra costs of educating LEP students. (Judge Marquez's Order for a Cost 
Studyy October 12, 2000, p. 1) 

2. Flores v. Arizona Consent Order 

This document sets forth the LEP Determination Criteria for students in 
Arizona schools, with clearly delineated responsibilities of the Superinten- 
dent of Public Instruction and of the State Board of Education. Mainly this 
document requires the selection of tests of English proficiency, cut scores 
on each test, monitoring LEP students after reassessment, circumstances 
applying to the writing of LEP Individual Education Plans, and the guide- 
lines for bilingual education and English language instruction. 
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3. Memorandum by Analizabeth R. Doan, 
Bilingual/Curriculum Director, Nogales Unified School 
District, August 19, 1999 

Since Ms. Doan is no longer on the staff of the Nogales Unified School 
District, it was not possible to obtain a full explanation of the Lau plan for 
the district. This document appears to be the testimony of Ms. Doan or the 
submission of a written memorandum in connection with the Flores v, Ari- 
zona case. The main points made are the following: 

A. The State is not providing sufficient funding to meet the educational 
needs of LEP students since costs are much higher in districts with very 
high percentages of such students (as is Nogales) compared to districts 
with small numbers of LEPs. An allegation was made that this inade- 
quate funding amounted to a case of discrimination against some dis- 
tricts, i.e., Nogales. 

B. Various points are made by various expert witnesses as to whether there 
is discrimination in requiring Hispanic students to pass the high school 
AIMS and SAT-9 tests; the elements necessary for an LEP program that 
adequately identifies, services, and monitors LEP students and provides 
access to an equal educational opportunity, and the costs involved. 
{NUSD No. ly Agenda Item Summary y Flores v. Arizona U.S. District Court 
Casey Discussion Itemy Analizabeth R. Doan, Bilingual/Curriculum Di- 
rector, August 19, 1999) 

4. Report from U.S. Department of Education to Dr. Raul 
Bejarano, Superintendent, Nogales, September 14, 1994 

Apparently there has been an Office for Civil Rights investigation of the 
Nogales Unified School District in the past and this document may have 
signaled the conclusion or near conclusion of the matter. Nogales signed a 
Corrective Action Agreement on May 24, 1993, and subsequently provid- 
ed sufficient documentation and corrections in “modifying some aspects of 
the District s approved plan for serving LEP students, and to demonstrate 
alternative language program (ALP) implemented during school year (SY) 
1993-94.” The report goes on to state, “This letter contains OCR’s analysis 
of the District’s plan and program implementation through SY 1993-94, 
numbered to correspond with each item of the Agreement. OCR found no 
compliance deficiencies and is requiring no additional District response in 
areas not listed below.” (Letter to Dr. Raul Bejarano, Superintendent, 
NUSD #1, from M. Arnold Chavez, Branch Chief, Compliance Enforce- 
ment Division II, Office for Civil Rights, U.S. Department of Education, 
September 14, 1994, p. 1) 

At the end of this report, the following commendation is given: “OCR 
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concluded that the District has established and implemented acceptable 
comprehensive plans and pirocedures for providing educational services to 
all LEP students, in accordance with its Agreement. We commend the 
District’s entire staff and School Board for the creativity and commitment 
they have demonstrated in developing and implementing programs and 
standards which will ensure quality educational opportunities to all stu- 
dents in the district.” (Letter, p. 12) The conclusions from the Office of 
Civil Rights gave Nogales very high praise indeed. 

Part A: General Background 
on Nogales Unified School District 

1. Description of Nogales and 
Overview of Schools and Programs 

Nogales is located approximately 70 miles south of Tucson, right on the 
border with Nogales, Sonora, Mexico. Nogales, Arizona, is a small rural 
community of approximately 22,000 inhabitants. Approximately 400,000 
people live in Nogales, Mexico. 

The main industry in Nogales, Arizona, is its base as a transportation 
center (trucking) for produce grown in Mexico and distributed in the 
United States. Recent changes in legislation will make it possible for 
Mexican trucks to travel in the U. S. and distribute produce directly to 
major markets. It is unclear how these new laws will affect the Nogales 
community. 

The public school system also plays a large role in the life of Nogales, 
both as the education system and as a major employer. There are six ele- 
mentary schools, two middle schools, one traditional high school, and one 
alternative high school. Approximately 6,000 students are served by these 
schools. The vast majority of these students are of Hispanic background, 
the overwhelming number being Mexican- Americans (> 95 percent). Most 
students speak Spanish as their first language, and many know very little 
English when they enter Nogales public schools. 

Contrary to conditions in many border towns, and many stereotypes 
about border tovms, the stability of the Nogales population is remarkably 
high. Nine out of ten students attending Nogales schools were born in the 
United States, and most are natives of this very tovm. The low mobility rate 
of the population is an extremely important factor in understanding the 
character of the public schools. In other cities, where student mobility is 
high, it is very difficult to craft educational improvements that will offset 
the disruptive effects of moving from place to place. In Nogales, mobility is 
not an issue. 
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The important demographic features are listed in Table IV-1, which 
identifies the school, the number of students at each school, the percent of 
students born in the U. S., the percent of LEP students, and the percent of 
students participating in the free or reduced lunch program, an accepted 
indicator of family socioeconomic status (SES). 



Table IV-1. 



Nogales Unified School District Demographics 



School 


Number of 


Percent 


Percent 


Percent 




Students 


U.S. Born 


LEP Low 


SES 


Elementary Schools 








. 


Bracket 


272 


95% 


81% 


88% 


Challenger 


604 


90% 


83% 


72% 


Coronado 


710 


90% 


65% 


59% 


Lincoln 


432 


91% 


91% 


94% 


Mitchell 


628 


90% 


85% 


96% 


Welty 


403 


94% 


73% 


89% 


Middle Schools 










Desert Shadows 


792 


88% 


72% 


63% 


Carpenter 


649 


90% 


78% 


88% 


High Schools 


Nogales High 


1636 




73% 


55% 


Santa Cruz Alternative 


152 


90% 


98% 


80% 


OVERALL 


6438 


90% 


80% 


78.4% 



A number of data indicators in Table IV-1 characterize patterns in 
Nogales schools that surfaced repeatedly in our talks with administrators 
and teachers. One of the most interesting findings is that not only a large 
majority of students but also of staff are natives of Nogales. All but one of 
the six principals, and all of the teachers and instructional assistants we 
interviewed, were born in Nogales. School personnel, therefore, are closely 
connected to the community and the schools, and they made it abundant- 
ly clear that they want the best programs available for their students. 

The high percentage of LEP students is a second noteworthy finding. 
There are at least two things about this figure that need to be understood 
to get a better sense of the community and how the schools operate. The 
first is that Nogales is a community where Spanish is more commonly used 
in the community than English. Virtually everyone we talked to about liv- 
ing in Nogales indicated that it is possible, and actually quite easy, to pur- 
sue a normal life in Nogales while speaking only Spanish. Spanish is the 
language used in most homes in Nogales, nearly every business has employ- 
ees highly proficient in Spanish, and there are a number of Spanish lan- 
guage television and radio stations, and Spanish print materials. Educators 
we talked to said this is what accounts for the high percentage of students 
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with a limited fluency in English when they enter kindergarten. Clearly 
having spent all their years in the U. S. they have been exposed to English, 
but their mastery of the language is not sufficient to give these children the 
readiness for academic learning in English, without some type of special- 
ized program support. 

Through elementary school, LEP status is based on standardized lan- 
guage test scores and the normal pattern is that as students move from 
grades K-5, more and more are reclassified as English proficient. In middle 
school, however, the classifications are less clear. At the Wade Carpenter 
Middle Academy, only 91 students are reported to be without English lan- 
guage proficiency, many having recently arrived from Mexico. But a far 
larger number are reported as LEP (506), and these students are identified 
by their score on the district wnriting test. Students who score below a 4, on 
a 1-6 scale, are classified as LEP, no matter what their fluency or compre- 
hension in English. 

2. Programs for LEP Students in the District 

According to materials given to us by the central administration of the 
Nogales Unified School District, there are 11 programs for LEP students 
designed to meet their needs as English-language learnersfThese are the 
program titles and definitions as provided by the District: Later in the 
report we describe each school’s program in greater detail. 

Nogales Unified School District #1 
Language Education Program Classifications 

A. K-6 Transitional bilingual program (#1) 

7-12 Stmctured bilingual program (#2) 

1. Early exit transitional bilingual education — students are in a bilingual 
program three years or less, or up to third grade (if in program since 
kindergarten). The goal is that the student be transitioned into an all- 
English class as quickly as possible. 

2. Late exit transitional bilingual education — students are in a bilingual 
program for more than three years, are up to fourth or fifth grades (if 
in program since kindergarten). The goal is that the student be tran- 
sitioned into an aU-English class by the end of fifth grade. 

B. K-12 Bilingual- bicultural program 

3. One-way developmental language education program — Spanish 
dominant English learners are instmcted in both English and Spanish 
throughout elementary grades. The goal is that both academic 
English and Spanish are developed. 
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4. Two-way developmental language education program — both English 
and Spanish dominant students are instructed in both English and 
Spanish throughout elementary grades. The goal is that both aca- 
demic English and Spanish are developed. 

C. English as a second language program 

5. ESL pull out — students are taken out of the mainstream instruction- 
al program and taught English. 

6. Content ESL — students are learning English using content subject 
areas. 

7. Sheltered English — all instruction (abstract and academic language 
along with content) is made comprehensible through the use of 
guarded vocabulary, visuals, cooperative learning, and hands-on activ- 
ities, while teacher checks frequently for understanding and student 
learning. 

8. ESL self-contained — students are homogeneously grouped and not 
instructed together wdth mainstream students, most of the day. 

D. No program 

9. Language Education Individual Education Plan (LEIEP) — because 
of a parent request to the principal, student achievement is being 
monitored using NUSD’s Language Education lEP. 

10. Mainstreamed English Learner — student is not in any of the above- 
mentioned programs. 

E. Special Education Accommodations 

11. Special Education Individual Education Plans (SEIEP) — the stu- 
dent has been identified as requiring special education services and 
his/her language education program has been outlined in his/her 
special education lEP. 

It must be said at the outset that we did not find eleven distinct programs 
in the schools; that in our professional opinion there are really no more than 
three or four identifiable programs in place; and that we were informed by 
administrators that the above list is due for streamlining very soon to reflect 
the realities in Nogales. 

The efficient way to describe the program types being implemented in 
Nogales is to list the schools by name that are providing the same recog- 
nized models, as follows: 



^‘1 A 
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Elementary Level 

Challenger, Coronado, Lincoln, and Mitchell Schools use structured 
English immersion programs (# 5, 6, and 7 above) for virtually all of their 
LEP students. 

Welty School uses an early-exit transitional bilingual education program 
(#1) for over half of its students in grades 1 and 2, and some strategies from 
#5, 6, and 7 in grades 3-5. It is in grade 3 that LEP students begin the grad- 
ual transition into the regular mainstream program. 

Bracker School has a more complex program which includes elements of 
early and late exit bilingual instruction (#1 and 2), combined with a two- 
way bilingual education program (#4), as well as regular mainstream class- 
rooms taught in English. The goal at the school is for students to be fully 
proficient in two languages at the end of elementary school. 

Secondary Level 

At the middle schools and the Nogales High School, there is a straightfor- 
ward ESL program that embodies elements of #5, 6, and 7. All the schools 
that are implementing English immersion programs also provide Spanish- 
language as a subject to some degree, except the Mitchell School which did 
not provide it this year but plans to do so next year. The Santa Cruz Altern- 
ative High School has a special program that offers more content and liter- 
acy support in Spanish than the other secondary schools. 

3. Superintendent’s District Goals for 2000-2001 

On October 16, 2000, a set of District Goals for the 2000-2001 School 
Year was adopted by the Nogales Unified School District, including a 
Statement of Commitment by the Governing Board, as follows: 

District Mission and Belief Statement 

By working as a team, within a positive environment all students will be 
successful, responsible, culturally appreciative, critical thinkers now and 
through the 21st century. 

Goal 7.' 

To align the curriculum and instruction at all school sites, both horizontal- 
ly and vertically, to the state standards, AIMS, and Stanford 9. 

Goal 2: 

To have most students achieving higher than average at all grade levels on 
the AIMS and Stanford 9. 

Goal3: 

Decrease by 20 percent the exemption rates for Limited- English Proficient 
students on all standardized tests. 
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Goal 4: 

That by 2001-2002 classrooms will have student- to- teacher ratios of 22:1 
at K-2; 25-1 at 3-5; and 28:1 at 6-12. 

Statement of Commitment 

The Governing Board is committed to working toward bet- 
ter teacher salaries and benefits^ and the recruitment and 
retention of qualified professionals. 

Nogales Unified School District is in a state of major transition this 2000- 
2001 school year. Dr. Kelt Cooper, the new superintendent, took office in 
August 2000, as did Ms. Anna Rosas who was appointed at about the same 
time to assume the duties of the Bilingual Education Program Coordinator 
who had left the district. Her position is now called “Assessment Director,” 
reflecting one of the major responsibilities of her office as the administra- 
tion of entry level and other tests to students enrolling in the district 
schools, to determine their status as possible English Language Learners. 
(The terms “Limited-English Proficient [LEP]” and “English language 
learners” are used interchangeably in this report.) 

Other changes in administrative staffing include the appointment of Dr. 
Marcelino Varona as Assistant Superintendent of Facilities (April 2001), 
and a search for an Assistant Superintendent for Curriculum and Instruc- 
tion that is not yet complete. New principals have been appointed this year 
from within the district to the Challenger and Mitchell schools, two of the 
six elementary schools, as well as an interim principal for Nogales High 
School. The Mitchell School is in its first year of a new, intensive English 
Acquisition Program. Replacement of the data entry specialist in the 
Assessment Office has occurred in recent months, with plans for a new pro- 
gram for recording student data being explored. 

In addition to the changes in leadership positions and the introduction 
of new system goals and policies, the Arizona state law on the education of 
English language learners changed dramatically with the passage of Propo- 
sition 203, the English for the Children initiative. A majority of Nogales 
citizens voted “Yes” on the initiative. This particular change should not be 
a cause for major disruption since four of the six elementary schools have 
been implementing structured English immersion programs for anywhere 
from 1 to 10 years, and the secondary schools have all consistently provided 
some form of English as a Second Language program for their LEP stu- 
dents. 

Taking all of the above conditions together, and considering the very 
short time frame allowed for the English Acquisition Program Cost Study, 
it is understandable that researchers faced serious difficulties in collecting 
precise data on the Nogales Unified School District. 
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Part B: 

Unique Conditions in Nogales Requiring 
Other Than Structured English 
Immersion, and Conclusions 

1. School Profiles 

Based on researchers’ on-site field work in Nogales, i.e., visits to five 
schools, interviews with administrators, observations of classrooms, and 
examinations of school documents, the following School Profiles were 
compiled to provide an overview of each school’s demographics, staffing, 
learning opportunities, and special features. 

Bracker Elementary 

Principal, Lucina A. Romero 

Grades: K - 5 : 

Number of students: 

272 - 81% are Limited-English Proficient (n = 219) on school entry 

Percent of students bom in US.: 95% 

Percent of students participating in breakfast/lunch program: 

Free 73%; reduced 15% 

First language ofLEP students: Spanish 

Types of programs for LEP students: 

Late exit transitional bilingual program, some elements of two-way bilin- 
gual instruction, sheltered English and content-based English as a Second 
Language (ESL). 

Assessments of Student Progress: 

Quarterly grades/report cards; annual review of Language Proficiency 
Quotient; district writing tests. 

Participation in State Assessments: 

One-third of LEP students (n = 70) take Aprenda Spanish tests in grades 
1-5; two-thirds (n = 150) take SAT-9; AIMS. 

Staff Endorsements: 

All staff are bilingual; all but one (music teacher) have full bilingual or 
ESL endorsement. 



COST Of ENGLISH ACQUISITION PROGRAMS 

77 





Staff Development: 

Weekly half-day early release for workshops, in-service; intersession train- 
ing opportunities. 

Extended learning opportunities for LEP students: 

Intersession classes three times/year; smaller class sizes; after-school pro- 
grams; summer programs. 

Challenger Elementary 

Principal, Norma Ahumada 

Grades: K - 5 
Number of students: 

604 - 83% are Limited- English Proficient (n = 501) on school entry 

Percent of students bom in US.: 90% 

Percent of students participating in breakfast/lunch program: 

Free 62%; reduced 10% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

Innovative Language Program started in 1998; English Intensive instruc- 
tion from Kindergarten on, initial literacy in English; Spanish language 
instruction, grades 1-5, 30-45 minutes daily. 

Assessments of Student Progress: 

Annual review of Language Proficiency Quotient; district writing test; 
report card grades. 

Participation in State Assessments: 

All students tested in English on SAT-9 and AIMS with very few waivers 
(only for new enroUees). 

Staff Endorsements: 

8 teachers have bilingual endorsement, 1 provisional bilingual, 5 have 
ESL, 2 have provisional ESL, 12 have no language endorsement. 

Staff Development: 

Weekly half-day early release for workshops, in-service; intersession train- 
ing opportunities. 

Extended learning opportunities for LEP students: 

Intersession classes three times/year; after-school ESL programs; SPED 
preschool for 3-5 year old at-risk children. 
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Coronado Elementary 

Principal, Annette Barber 

Grades. K - 5 
Number of students: 

710 - 65.3% are Limited- English Proficient (n = 464) on entry 

Percent of students bom in U.S,: 90%, mostly in Nogales area 

Percent of students participating in breakfast/lunch program: 

Free and reduced 59% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

Intensive English instruction, literacy and content-based; initial reading in 
English; Spanish as a foreign language, daily, grades 1-5. 

Assessments of Student Progress: 

Teacher assessments/report cards; annual review of Language Proficiency 
Quotient; district writing test. 

Participation in State Assessments: 

All students take SAT-9, AIMS in English, very few (only new arrivals) 
given waivers. 

Staff Endorsements: 

4 have bilingual endorsements, 10 have ESL endorsements, 6 have 
provisional ESL, and 9 have no language endorsements. 

St^ Development: 

Weekly half-day early release for workshops, in-service; intersession 
training opportunities. 

Extended learning opportunities for LEP students: 

Intersession classes three times/year; after-school programs on early 
release days. 
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Lincoln Elementary 

Principal, Liza Montiel 

Grades: K - 5 
Number of students: 

432 - 92% are Li mi ted- English Proficient (n = 394) on school entry 

Percent of students bom in U.S.: 91% 

Percent of students participating in breakfast/lunch program: 

Free 87%; reduced 7% 

First language of LEP students: Spanish 

Types of programs for LEP students: ESL 

Assessments of Student Progress: 

Teacher grades/report cards; annual review of Language Proficiency 
Quotient; district writing tests. 

Participation in ^tate Assessments: 

AIMS and Stanford 9 testing in English — all students. 

Staff Endorsements: 

6 teachers have bilingual endorsement, 5 have ESL endorsement, 1 has 
provisional ESL, and 10 have no language endorsement. 

Staff Development: 

Teacher training workshops and Compliance Consultant reports at staff 
meetings. 

Extended learning opportunities for LEP students: 

After-school programs for English and Reading; Intersession classes 
offered. 

A. J. Mitchell Elementary 

Principal, Angelina Johnson 

Grades: K - 5 
Number of students: 

628 - 85% are Limited-EngUsh Proficient (n= 536) on school entry 

Percent of students bom in U.S.: 90%, mosdy in Nogales area 

Percent of students participating in breakfast/lunch program: 

Free and reduced 96% 

First language of LEP students: Spanish 
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Types of programs for LEP students: 

English Language Acquisition Program started 1 year ago; initial reading 
in English; pull-out ESL tutoring for beginners; no Spanish instruction 
this year; Spanish language instruction to begin next year in grades 3-5. 

Assessments of Student Progress: 

Teacher grades and report cards; annual review of Language Proficiency 
Quotient; district/state writing assessments. 

Participation in State Assessments: 

All students take SAT-9 and AIMS tests in English, with very few 
exceptions for new arrivals. 

Staff Endorsements: 

1 teachers have bilingual endorsement, 11 have ESL endorsement, and 11 
have no language endorsement. 

Staff Development: 

Weekly half-day early release for workshops, in-service; intersession 
training opportunities. 

Extended learning opportunities for LEP students: 

Intersession classes three times/year; SPED preschool for 3-5 year old at- 
risk children. 

Mary Welty Elementary 

Principal, Javier Barajas 

Grades: K - 5 
Number of students: 

403 - 73% are Limited-English Proficient (n = 293) on school entry 

Percent of students bom in US.: 94%, mostly in Arizona 

Percent of students participating in breakfast/lunch program: 

Free and reduced 89% 

First language of lEP students: Spanish 

Types of programs for LEP students: 

Transitional bilingual program, English-language development in K; 
Spanish literacy in grades 1-3, with ESL; transition to English reading 
begins in grade 3, continues through grades 4 and 5. 

Assessments of Student Progress: 

Quarterly grades/report cards; annual review of Language Proficiency 
Quotient; district writing tests. 



COSl OF ENGLISH ACQUISITION PROGRAMS 

SI 





Participation in State Assessments: 

LEP students enrolled in Nogales schools less than 3 years take Aprenda, 
exempt from SAT-9; AIMS in English. 

Staff Endorsements: 

7 teachers have bilingual endorsement, 9 have ESL endorsement, 1 has 
provisional ESL, and 4 have no language endorsement. 

Staff Development: 

Weekly half-day early release for vs^orkshops, in-service; intersession train- 
ing opportunities. 

Extended learning opportunities for LEP students: 

After-school Reading and Homework Clubs; 3rd, 4th and 5th grade pull- 
out program v^th a retired volunteer teacher; Intersession classes; 
Newcomer Program. 

Desert Shadows Middle School 

Principal, Mark Valenzuela 

Grades: 6-8 
Number of students: 

792 - 72% are Limited- English Proficient (n = 572) on entry 

Percent of students bom in US.: 88% 

Percent of students participating in breakfast/lunch program: 

Free 54%; reduced 9% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

ESL and Sheltered English to develop English language and literacy in 
English, 1 block of Spanish for some students each semester. 

Assessments of Student Progress: 

Report cards; annual review of Language Proficiency Quotient; district 
writing tests. 

Participation in State Assessments: 

AIMS and Stanford 9 tests administered in English — all students. 

Staff Endorsements: 

8 teachers have bilingual endorsement, 1 has provisional bilingual, 6 have 
ESL endorsement, 1 has provisional ESL, 18 have no language endorse- 
ment. 
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Staff Development: 

Maria Montano-Harmon Training; Compliance Consultants report to 
staff. 

Extended learning opportunities for LEP students: 

Before- and after-school tutoring; intersession classes and summer school. 

Wade Carpenter Middle Academy 

Principal, Rebecca Holler 

Grades: 6-8 
Number of students: 

649 - 78% are counted as Limited- English proficient (n = 506)* 

Percent of students bom in U.S.: 90% 

Percent of students participating inbreakfast/lunch program: 

Free and reduced 88% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

ESL Program to develop English language and literacy 2 blocks daily — 
Humanities ESL and Social Studies ESL; 1 block Spanish Language and 
Literacy for all LEP students alternate semesters. 

Assessments of Student Progress: 

Quarterly grades/report cards; annual review of Language Proficiency 
Quotient; district writing test. 

Participation in State Assessments: 

From 85 - 100% of eligible students take the SAT- 9 and AIMS tests 
(only in grade 8) in English. 

Staff Endorsements: 

8 teachers have bilingual endorsements, 2 have provisional bilingual; 10 
have ESL endorsement, 6 have provisional ESL, and 12 have no language 
endorsement. 

Staff Development: 

Training provided in ESL/Sheltered English; ADE training in assisting 
LEP students in meeting state standards and the needs of second- 
language learners; new teachers given release time to observe experienced 
teachers in Sheltered English classes; teachers attend workshops and upon 

*Only 91 students are reported to be without proficiency in English for regular classroom work 
(most having recendy arrived from Mexico). The rest who make up the LEP number are stu- 
dents who are proficient in English but are below level on the writing tests. 
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return make presentations to the staff. 

Extended learning opportunities for LEP students: 

Intersession classes four times/year; mandatory, extended-day remedial 
programs; parent/community program teaching skills to help parents sup- 
port student learning efforts. 

Nogales High School 

Principal (Interim), Karen Conley 

Grades: 9-12 
Number of students: 

1636 - 73.9% are Limited-English Proficient (n = 1193) 

Percent of students bom in U.S.: 90% 

Percent of students participating in breakfast/lunch program: 

Free 48% and reduced 7% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

ESL Program to develop English language and literacy, 2 blocks daily. 

Assessments of Student Progress: 

Quarterly grades/report cards; district writing tests. 

Participation in ^tate Assessments: 

Eligible students (3 years or longer in U.S.) take SAT-9 and AIMS tests 
in English. 

^tajf Endorsements: 

14 teachers have bilingual endorsement, 14 teachers have ESL endorse- 
ment, 1 teacher has provisional ESL, and 46 teachers have no language 
endorsement. 

^tajf Development: 

Teacher training and conferences for AIMS preparation, reading, writing, 
math, study skills — all faculty, September 2000-June 2001; workshop on 
standard reading techniques for LEP students; workshop on graphic 
organizers to use with sheltered reading assignments. 

Extended learning opportunities for LEP students: 

AIMS prep in reading, writing, math — 8th period class for sophomores 
and juniors supported by Title VII grant; free tutoring for all NHS stu- 
dents on Saturdays, January- April 2000, on English, reading, writing, 
study skills/test taking skills. 
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Santa Cruz Alternative High School 

Principal, Mike Hart 

Grades: 9-12 
Number of students: 

156 - 98% are Limited- English Proficient (n = 153) 

Percent of students bom in U.S.: 90% 

Percent of students participating in breakfast/lunch program: 

Free 75% and reduced 5% 

First language of LEP students: Spanish 

Types of programs for LEP students: 

Students from Nogales HS and other Arizona high schools enter ESL 
programs for 2 nine- week semesters, some stay longer, some test out 
sooner; special grant specifies goal to be a Bilingual Program High 
School. 

Assessments of Student Progress: 

Report cards; progress reports; criterion-referenced tests; AIMS in 
English; SLEP; Aprenda; diagnostic tests; computer assessments. 

Participation in State Assessments: 90% in English; 10% in Spanish. 

Staff Endorsements: 

3 teachers have bilingual endorsements, 3 have ESL endorsements, 1 has 
provisional ESL, and 4 have no language endorsements. 

Staff Development: 

Ongoing training: biliteracy, identifying LEP student needs, assessing 
LEP students, multiculturalism, language acquisition. 10 teachers attend- 
ed the Phoenix NABE in March 2001. Teachers’ tuition paid to take 
courses towards endorsement. 

Extended learning opportunities for LEP students: 

After-school programs; tutorials; content reinforcement in Spanish; alter- 
native classroom presentations and assessments in Spanish, based on stu- 
dent choice; summer school. 
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2. Performance Data on Stanford 9 Tests-1999 and 2000 

In the following table we show selected data on the performance of chil- 
dren in Nogales schools over the last two years, in 1999 and 2000, There 
are inconsistencies in the way data were organized and reported in the 
sources we accessed to try to gain a representative picture of student out- 
comes in reading, math, and language. We used data available on the 
Arizona Department of Education website, and local data organized by the 
Nogales school district. Because of data inconsistencies we will not make 
many inferences. One inconsistency is that sometimes the data are report- 
ed according to the number of students who take a particular test; some- 
times the same information is represented in terms of the percentage of stu- 
dents who took the test. These are not the same things, obviously, and it 
would be helpful if future reporting formats were standardized to include 
both the number of students tested and the percentage of students tested. 

To estimate improvements in student performance, it is probably best to 
view the following table in terms of overall percentile level, and differences 
between schools at each of the two years. In the elementary grades, in both 
1999 and 2000, students tend to score below the 50th percentile, except for 
Coronado school, where scores tend to be around the 50th and 60th per- 
centiles. When schools do not test close to 100 percent of their students 
(i.e., Robert Bracker and Lincoln), the scores tend to be higher. It is clear 
Bracker might not test some students on the SAT-9 because many students 
are likely to take the test in Spanish for up to three years. It is not clear why 
Lincoln would not test aU of its students, however, since they have an 
immersion program. Arizona law allowed exemptions from standardized 
testing in English for LEP students who are in school less than three years 
until the passage of Proposition 203 in November 2000. 

Overall, the patterns of achievement are similar on reading, language, 
and math. Also, scores tend to be just slightly lower in middle school and 
high school, except at the alternative high school, where they are quite a bit 
lower. Generally, scores in Nogales are in the low-average range, except at 
Coronado, where they tend to be in the average range. It will be important 
to see how scores change over the course of the next few years, as programs 
change and become established and as reporting formats improve. 
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Table IV-2. 

Nogales Unified School District 
Elementary Schools Stanford 9 Test Scores - READING 





Year 1999 




Year 2000 


School & Grade 


% Students 


Percentile 


# of Students 


Percentile 




Tested 


Rank 


Tested 


Rank 


AJMitcheU 


Grade 2 


70 


23 


94 


30 


Grade 3 


100 


29 


71 


31 


Grade 4 


100 


23 


65 


31 


Grade 5 


100 


24 


85 


39 


Challenger 


Grade 2 


100 


31 


31 


64 


Grade 3 


100 


30 


40 


58 


Grade 4 


100 


35 


41 


59 


Grade 5 


100 


31 


73 




Robert Bracker 


Grade 2 


50 


46 


12 


69 


Grade 3 


78 


45 


6 


36 


Grade 4 


64 


60 


30 


39 


Grade 5 


100 


32 


33 


38 


Francisco Vasquez 


de Coronado 








Grade 2 


100 


54 


35 


68 


Grade 3 


100 


45 


55 


67 


Grade 4 


100 


55 


44 


69 


Grade 5 


100 


50 


77 




Lincoln 


Grade 2 


21 


63 


21 


27 


Grade 3 


14 


• 


10 


43 


Grade 4 


16 


54 


30 


40 


Grade 5 


100 


22 


56 


25 


MaryWelty 


Grade 2 


• 


• 


7 


34 


Grade 3 


45 


46 


9 


26 


Grade 4 


100 


26 


18 




Grade 5 


100 


33 


25 


28 



* = Missing Data 
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Table IV-3. 

Nogales Unified School District 
Middle and High Schools Stanford 9 Test Scores - REIADING 

Year 1999 Year 2000 

School & Grade % Students Percentile # of Students Percentile 

Tested Rank Tested Rank 

Santa Cruz Alternative 

Grade 9 61 16 ^ 

Grade 10 78 15 38 

Grade 11 56 16 19 

Wade Carpenter Middle 



Grade 6 


100 


24 


170 3 


Grade 7 


100 


24 


133 2 


Grade 8 


100 


29 


149 3 



Desert Shadows Middle 

Grade 6 100 35 ^ 

Grade? 98 35 213 

Grade 8 100 43 195 



Nogales High 


Grade 9 


93 


27 


499 


29 


Grade 10 


100 


29 


336 


32 


Grade 11 


100 


34 


195 


37 


* = Missing Data 








Table IV-4. 

Nogales Unified School District 
Elementary Schools Stanford 9 Test Scores - LANGUAGE 







Year 1999 


Year 2000 


School & Grade 


% Students 
Tested 


Percentile 

Rank 


# of Students 
Tested 


Percentile 

Rank 


AJMitcheU 


Grade 2 


73 


12 


94 


27 


Grade 3 


100 


36 


70 


34 


Grade 4 


100 


25 


65 


33 


Grade 5 


100 


20 


85 


39 


Challenger 


Grade 2 


100 


25 


31 


53 


Grade 3 


100 


45 


40 


73 


Grade 4 


100 


39 


40 


58 


Grade 5 


100 


28 


73 


36 


Robert Bracker 


Grade 2 


50 


34 


12 


56 


Grade 3 


78 


59 


7 


44‘ 


Grade 4 


64 


55 


36 


3f 


Grade 5 


100 


25 


35 


40 


Francisco Vasquez 


de Coronado 








Grade 2 


100 


47 


46 


57 


Grade 3 


100 


52 


55 


76 


Grade 4 


100 


54 


44 


74 


Grade 5 


100 


58 


85 


49 


Lincoln 


Grade 2 


21 


39 


27 


20 


Grade 3 


14 


* 


10 


47 


Grade 4 


16 


51 


31 


40 


Grades 


100 


21 


55 


29 


MaryWelty 


Grade 2 


40 


* 


6 


22 


Grade 3 


43 


70 


10 


47 


Grade 4 


100 


32 


18 


59 


Grade 5 


100 


32 


29 


28 



* = Missing Data 
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Table IV-5. 

Nogales Unified School District 

Middle and High Schools Stanford 9 Test Scores - LANGUAGE 

Year 1999 Year 2000 

School & Grade % Students Percentile # of Students Percentile 

Tested Rank Tested Rank 

Nogales High 

Grade 9 91 27 488 31 

Grade 10 100 32 335 36 

Grade 11 100 37 195 40 

Santa Cruz Alternative 

Grade 9 64 13 26 1 

Grade 10 75 16 39 1 

Grade 11 56 24 18 1 



Wade Carpenter Middle 



Grade 6 


100 


20 


173 


Grade 7 


100 


28 


132 


Grade 8 


100 


24 


152 



Desert Shadows Middle 
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Table IV- 6. 

Nogales Unified School District Elementary Schools 
Stanford 9 Test Scores - MATHEMATICS 





Year 1999 




Year 2000 


School & Grade 


% Students 


Percentile 


# of Students 


Percentile 




Tested 


Rank 


Tested 


Rank 


AIMitcheU 


Grade 2 


74 


18 


95 


49 


Grade 3 


100 


33 


70 


32 


Grade 4 


100 


28 


71 


35 


Grade 5 


100 


21 


86 


48 


Challenger 


Grade 2 


100 


39 


32 


79 


Grade 3 


100 


44 


43 


58 


Grade 4 


100 


50 


43 




Grade 5 


100 


43 


73 


44 


Robert Bracker 


Grade 2 


50 


28 


13 


62 


Grade 3 


81 


49 


8 


44 


Grade 4 


70 


57 


38 


46 


Grade 5 


100 


33 


35 


* 


Francisco Vasquez 


de Coronado 








Grade 2 


100 


74 


46 


72 


Grade 3 


100 


54 


55 


79 


Grade 4 


100 


69 


44 


82 


Grade 5 


100 


74 


84 


67 


Lincoln 


Grade 2 


23 


49 


27 


58 


Grade 3 


14 


* 


10 


46 


Grade 4 


17 


65 


31 


59 


Grades 


100 


30 


56 


35 


MaryWelty 


Grade 2 


* 


* 


6 


44 


Grade 3 


43 


46 


9 


29 


Grade 4 


100 


26 


18 


71 


Grade 5 


100 


33 


29 


23 


* = Missing Data 
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Table IV-7. 

Nogales Unified School District 

Middle and High Schools Stanford 9 Test Scores-MATHEMATICS 



Year 1999 Year 2000 



School & Grade 


% Students 
Tested 


Percentile 

Rank 


# of Students 
Tested 


Percentile 

Rank 


Nogales High 


Grade 9 


92 


36 


500 


39 


Grade 10 


100 


33 


337 


38 


Grade 11 


100 


34 


199 


41 


Santa Cruz Alternative 


Grade 9 


64 


18 


27 


23 


Grade 10 


75 


20 


39 


21 


Grade 11 


59 


20 


18 




Wade Carpenter Middle 


Grade 6 


100 


38 


172 


44 


Grade 7 


100 


35 


129 


40 


Grade 8 


100 


36 


152 


38 



Desert Shadows Middle School 
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Table IV-8. 

Nogales Unified School District 
Elementary Schools AIMS Results - Spring 2000 





Reading 




Writing 




Mathematics 


School & Grade 


#of 

Students 

Tested 


Score 


#of 

Students 

Tested 


Score 


#of 

Students 

Tested 


Score 


AIMitcheU 


Grade 3 


67 


500 


67 


501 


65 


468 


Grade 5 


89 


500 


90 


479 


82 


486 


Challenger 


Grade 3 


39 


522 


39 


547 


39 




Grade 5 


73 


492 


72 


477 


71 


466 


Francisco Vasques 


de Coronado 












Grade 3 


53 


535 


54 


554 


55 


528 


Grade 5 


83 


510 


85 


511 


87 


525 


Lincoln 


Grade 3 


10 


514 


11 


511 


11" - 


478 


Grade 5 


15 


501 


16 


485 


13“- 


496 


MaryWelty 










- 




Grade 3 


40 


484 


42 


497 


44- 


447 


Grade 5 


42 


496 


25 


465 


50" 


456 


Robert Bracker 














Grade 3 


5 


* 


5 




5 


* 


Grade 5 


35 


509 


35 


486 


35 


491 


‘Missing Data 



Table IV-9, 

Nogales Unified School District 
Middle and High Schools AIMS Test Results - Spring 2000 





Reading 




Writing 




Mathematics 


School & Grade 


#of 

Students 

Tested 


Score 


#of 

Students 

Tested 


Score 


#of 

Students 

Tested 


Score 


Desert Shadows Middle 


Grade 8 


190 


485 


186 


462 


190 


461 


Wade Carpenter Middle 


Grade 8 


166 


482 


166 


456 


169 


452 


Nogales High 


Grade 10 


345 


499 


348 


461 


350 


423 


Grade 11 (retest) 


166 


481 


215 


455 


250 


427 


Santa Cruz Alternative 


Grade 10 


16 


458 


15 


417 


15 


397 


Grade 11 


24 


458 


25 


421 


26 


366 
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Table IV- 10. 

Nogales Unified School District 
Elementary Schools - Arizona Measure of Academic Progress 






1998-1999 


1999-2000 


School & Grade 


Reading 
OYG (SR) 


Math 
OYG (SR) 


Reading 
% Exp. Gain 


Math 

% Exp. Gain 


Challenger 


Grade 2-3 


- 


100 


101 




Grade 3-4 


Yes (3) 


Yes (2) 


125 


142 


Grade 4-5 


No (2) 


Yes (3) 


7 


44 


AJMitcheU 


Grade 2-3 


- 


- 


116 


130 


Grade 3-4 


Yes (2) 


No (1) 


98 


78 


Grade 4-5 


No (2) 


No (1) 


168 


160 


Francisco Vasquez 


de Coronado 








Grade 2-3 


- 


- 


68 


93 


Grade 3-4 


Yes (5) 


Yes (5) 


149 


173 


Grade 4-5 


No (3) 


Yes (4) 


57 




Lincoln 


Grade 2-3 


- 


- 


- 


- 


Grade 3-4 


- 


- 


- 


- 


Grade 4-5 


No (4) 


Yes (3) 


- 


- 


MaryWelty 


Grade 2-3 


- 


- 


- 


- 


Grade 3-4 


Yes(l) 


Yes (4) 


- 


- 


Grade 4-5 


No(l) 


No (2) 


56 


40 


Robert Bracker 


Grade 2-3 


- 


- 


- 


- 


Grade 3-4 


Yes(l) 


Yes (5) 


105 


94 


Grade 4-5 


Yes (4) 


Yes (3) 


155 


141 



Note. OYG (One Year Growth): Yes = one or more years of growth based on national norms; 
no = less than one year of growth. SR = Star Rating 
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Table IV-11. 






Nogales Unified School District Middle and High Schools 
Measure of Academic Progress 




1998-1999 


1999-2000 


School & Grade 


Reading 
OYG (SR) 


Math 
OYG (SR) 


Reading 
%£xp. Gain 


Math 

% Exp. Gain 


Desert Shadows Middle 


Grade 5-6 


- 


- 


92 


8 


Grade 6-7 


No (2) 


No(l) 


66 


60 


Grade 7-8 


Yes (4) 


No (1) 


153 


156 


Wade Carpenter Middle 


Grade 5-6 


- 


- 


169 


283 


Grade 6-7 






93 


84 


Grade 7-8 




No (3) 


156 


82 



Note. OYG (One Year Growth): Yes = one or more years of growth based on national norms; 
no = less than one year of growth. SR = Star Rating 



3. Unique Conditions 

The unique conditions affecting the education of Nogales, students have 

been touched on in several places in this report. These are the specific cir- 
cumstances of English-language learners in the Nogales schools that are 

different from other districts: 

A. Nogales, Arizona, is an isolated community on the Mexican border and 
the nearest U.S. city is 70 miles away. 

B. Nogales, Sonora, Mexico, is the nearest city with a population of 
400,000, a city with which U. S. Nogales residents have much contact. 

C. The population is strongly homogeneous, i.e., the entire community of 
Nogales, with minor exceptions, is Spanish speaking. 

D. Nogales has a stable population with very little mobility, most residents 
and school children having roots in the community of more than one 
generation, including many of the school personnel. 

E. All elements of community life are predisposed to maintain fluency in 
the Spanish language, requiring extraordinary efforts by the schools to 
promote the English language for fluency, literacy, social, and academic 
purposes. 

R Parents of Nogales students are documented by school personnel as 
being in favor of a strong emphasis on English language instruction and 
promoting high academic performance in English. 
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4. Conclusion 

Considering these circumstances, and applying our own professional judg- 
ment, it is our firmly held conclusion that Structured English Immersion 
Programs can meet the needs of the students in the Nogales Unified School 
District. 

It is precisely because of the unique conditions enumerated above, i.e., 
geography, demographics, community stability, and parent preference, that 
programs emphasizing rapid, intensive focus on second language (English)' 
learning and literacy, combined with early introduction of subject matter 
taught in English with special strategies, appears to be the wisest course. 
This is the approach being implemented in four of the six elementary 
schools now, and will be the program in a fifth elementary school next year, 
as well as in all the secondary schools. 

It is essential to note here that this does not mean that the development 
of language skills in Spanish is neglected. As mentioned in Part B, number 
1, all schools (except Mitchell) currently offer daily instruction in Spanish, 
and Mitchell will be offering a course starting in the next school year. 

The Bracker Elementary School staff shares a philosophical belief in the 
efficacy of the traditional bilingual education approach. It is their belief that 
their students will perform best through a mostly late-exit bilingual instruc- 
tion approach, vrith language and literacy developed first in Spanish, from 
grades K through 3, and a gradual transition to English instruction begin- 
ning in grade 3 and continuing through grade 5. It is the school’s goal to 
develop equal skills in both languages by the end of grade 5, and this goal 
requires several years of instruction in two languages. Students at Bracker 
School are not different from all other elementary school students in 
Nogales, and we found no evidence suggesting that these students could 
not succeed through an English immersion program. Time constraints did 
not permit researchers to prove or disprove the success of the Bracker 
School approach. 

Overall, the population of students in Nogales schools is remarkably 
homogeneous and consistent from one school to the next. All elementary 
schools serve student populations that are mosdy from poverty back- 
grounds. The majority of students in all of the elementary schools enter 
kindergarten knowing very little English; at all schools over 90 percent 
begin school as English-language learners. Most Nogales schools have in 
fact been providing structured immersion programs for all of their students, 
and clearly would continue to do so even vrithout the pressure of the new 
law requiring schools to teach in English. It is interesting to note that the 
longer a school has provided such instruction, the higher the level of aca- 
demic performance its students are achieving. The Coronado School began 
its program over 10 years ago; Challenger School began 3 years ago; and 
these two schools are ranked first and second in academic achievement at 
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the elementary level. While it may be said that these two schools have a 
somewhat lower percentage of students from low SES families than other 
elementary schools (59 percent and 72 percent, respectively), these are still 
high proportions of children who are succeeding, relative to other Nogales 
students, in spite of poverty backgrounds. 

Paying heed to parental preference is essential to promoting a high level 
of support in the home for school programs. In all the schools that have 
moved to structured English immersion, there had been a former period 
when they implemented transitional bilingual education programs in 
Spanish. The principal of Coronado School was remarkably straightfor- 
ward about the reasons she and her staff believe that English should be 
taught from the first day of school to all of the students at Coronado: there 
was unanimity in their belief that it is critical for students to learn English, 
and to begin working hard on that goal from the first day of school, and 
that this is the approach that parents have unwaveringly requested. 

Norma Ahumada took the position of principal at the Challenger School 
one year ago, partly because she wanted to implement a structured English 
immersion program. As a former staff member at the Lincoln Elementary 
School, Ms. Ahumada and her colleagues witnessed the transformation 
from a transitional bilingual program to structured immersion and the 
improved results in student achievement. Challenger School explicitly fol- 
lowed Lincoln’s model in setting up its own program and teachers are very 
pleased with the early results. 

The Challenger staff is convinced that while it is easy and comfortable to 
know only Spanish in Nogales, students with academic ambitions and who 
may seek work beyond the boundaries of Nogales need a high level of 
English proficiency. Challenger staff felt that they gave transitional bilin- 
gual education their best effort and it had not resulted in students learning 
English to the level they need for school success. Parents, too, were becom- 
ing more and more vocal about wanting the school to teach their children 
English, to the point where they needed to be convinced of the benefits of 
a daily enrichment lesson in Spanish. More information supporting the 
appropriateness of English immersion programs for Nogales students 
appears in the School Profiles preceding this section. 

In an interview with the principal of the Mitchell School, Ms. Angelina 
Johnson, we were informed of a planning session taking place at the time 
of our visit. Three elementary school principals, Ms. Ahumada, Mr. Barajas, 
and Ms. Montiel, were working on a Structured English Immersion Imple- 
mentation Plan (Proposition 203). The plan calls for staff development activ- 
ities in the summer months, 2001, to prepare for full organization of the 
schools for the fall of that year. The plan is in the discussion stage, with, the 
possibility of two-hour English language arts blocks. Sheltered Math, 
Science, and Social Studies being considered, along with various ways of 
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providing integrated learning opportunities for LEP students with their 
English-dominant classmates. Evidendy the elementary school principals 
group is moving ahead with preparations for strengthening and expanding 
English immersion instruction. Whether the Bracker School will retain its 
current bilingual programs as a flexible option for parents requesting 
waivers is not known at present. 

5. Summary and Recommendations for Further Research 

In conclusion, we restate our affirmation of the special circumstances in 
Nogales and these are fully recognized by the qualified and experienced 
school personnel and by the new administrative team. While there may be 
some few students vtith special learning problems requiring special educa- 
tion referrals and very special assistance, the overall student population in 
Nogales is best served in the manner in which it is now being served, for 
the most part: by being provided with special help in second language 
learning and literacy from day one of school enrollment. That appears to be 
the agreed-on first priority in the district. An additional benefit for Nogales 
students is the enrichment lessons in Spanish from grades 1 through high 
school which, added to the prevalence of community homogeneity in the 
usage of Spanish, bodes well for maintaining the viability of the communi- 
ty language and culture. 

Considering the transitional nature of the 2000-2001 school year with 
substantial changes in the administrative leadership, an announced shift to 
a more centrally coordinated district, the possibility of overhauling the stu- 
dent data program, and the imperative to implement a new state law, it is 
our strong recommendation that the Nogales Unified School District 
design an internal school evaluation program to assess the effectiveness of 
new policies and programs at least for the next three years. For this endeav- 
or to be eflfective, it is advisable to improve the recording and analysis of test 
data, by school, to include the absolutely essential information on number 
and percentage of LEP students participating in the various assessment 
measures, for a realistic report of student achievement and improvement. 

It wUl be especially useful if Nogales schools adopt a common reporting 
format, and pursue a policy of excusing students from testing in English 
only when absolutely necessary (the most common example is students who 
have only just entered the U.S. and know no English at all). It is better prac- 
tice to test students in English, even when they have not had three years of 
English instruction, to confirm what their English performance level real- 
ly is, rather than speculating, since all children do not enter Nogales at the 
exact same level of second language proficiency. It is possible to test nearly 
all students but report the results in two separate categories by coding the 
test results: an unofficial reporting that gives aU scores including LEP stu- 
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dents (for internal use by teachers and administrators), and an official 
report, used for comparing schools to each other, but that does not include 
the scores for newly arrived LEP students and, therefore, does not penalize 
the school for low scores. It is the system currently in use in California. The 
unofficial data are valuable for teachers to assess all students’ progress and 
for schools to monitor growth from )^ear to year. 

Appendix IV-A 

Lists of Documents Examined, Nogales 
Personnel Interviewed, and Schools Visited 

Documents Examined 

Internal forms 

Language Education Program Classifications 

Language Education Program Compliance Mini-Audit, 2000-2001 Guidelines 
Identification of English Language Learners, Grades 2*and 12 
Reassessment Flow Chart 

Checklist for Developing English Learners’ Academic Competency 
Structured Interview, for students having a language background other than 
Spanish or English 

Multicultural Education Program, Teacher Record 
Student Profile - Test Results (sample) 

K-1 Parent Notification Letter 
2-5 Parent Notification Letter 
6-12 Parent Notification Letter 
Compliance Consultant’s Duties 
Compliance Deadlines, 2000-2001 
Reassessment and Reclassification Form 
2000-2001 Follow-Up of Reclassified Students 

Setting Language Proficiency Quotients - Reading, Writing, Oral Language 

Enrollment K-12 by Home School - 2000-2001 (updated 1/31/01) 

LAS Test Scores (Level 1 [NEP] to Level 5 [FEP]) - Numbers of students at 
each grade level, by school 

Stanford 9 Test Scores - By school, grade, average percentile rank. School Years 
1999 and 2000 

Lau and State Compliance Mini Audit Reports for each school, 1999-2000 

DRAFT - STRUCTURED ENGUSH IMMERSION IMPLEMENTATION 
PLAN (Proposition 203) 

Bracket School’s Language and Literacy Program; Descriptions of special lan- 
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guage, reading, and writing programs; Into English Program; Second Language 
Acquisition Teaching Methodologies; Comprehensive School Reform Design 
Grant 

Challenger School’s Innovative Language Program - An Interim Review of the 
Program and Its Progress 

Wade Carpenter Middle Academy - Life Strategies for Kids, Grant Performance 
Report, October 2000 

Legal Documents 

Judge Marques’s Order for a Cost Study, January 24, 2000 

Flores v. Arizona Consent Order - LEP Determination Criteria for Students in 
Arizona, rules adopted by the State Board of Education, December 31, 2000 

In re Flores v, Arizona - Memorandum by Analizabeth R. Doan, 
Bilingual/Curriculum Director, Nogales Unified School District, August 19, 1999 

Report from U.S. Department of Education, Office for Civil Rights, to Dr. Raul 
Bejarano, Superintendent, NUSD, September 14, 1994 



Nogales Staff Members Interviewed 

Dr. Kelt Cooper, Superintendent 

Anna Rosas, Assessment Director 

Lucina Romer, Bracker School Principal 

Michelle Orgui, Bracker School 

Norma Ahumada, Challenger School Principal 

Annette Barber, Coronado School Principal 

Angelina Johnson, Mitchell School Principal 

Myrna Alavez, Welty School Compliance Consultant 

Rebecca Holler, Wade Carpenter Middle Academy Principal 

Jan Sigurdson, Wade Carpenter Dean of Students 

Monica Jimenez, Assessment Department, Data Entry Specialist 



Schools Visited, April 3-4, 2001 

Bracket Elementary 
Challenger Elementary 
Coronado Elementary 
Welty Elementary 
Wade Carpenter Middle Academy 
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Testing English 
Language Learners for 
School Accountability 



Adapted from a Report Prepared for 
San Francisco Unified School District et al. 
V. State Board of Education et al. 



Susan E. Phillips 

Background Information 

The following sections include a career summary, a brief account of prior 
legal work, a description of my role as a consultant to the California State 
Board of Education and the California Department of Education, and a 
summary of the basis for the professional opinions expressed in this report. 

Career Summary 

Currendy, I am an independent psychometric and assessment law consult- 
ant for a number of state and district assessment programs. Previously, I was 
a member of the graduate faculty in the College of Education at Michigan 
State University for 18 years and taught courses in educational measure- 
ment with a specialization in legal and policy issues. Prior to joining the 
Michigan State University graduate faculty, I worked in the test division of 
Riverside Publishing and at the American College Testing Program in Iowa 
City, Iowa. My educational training includes a Ph.D. in educational meas- 
urement and statistics from the University of Iowa in 1981 and a law degree 
in 1990. 

My research and scholarship activities have included more than 60 pre- 
sentations at national professional meetings and 30 papers published in 
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nationally recognized measurement, policy, and education law journals. 
Topics have included standard setting, performance assessment, testing 
accommodations for persons with disabilities, modifications for English 
language learners, testing to award diplomas, the Golden Rule remedy, 
teacher licensure testing, and other issues in assessment law. I am Guest 
Editor and author of an article on Psychometric Issues for a forthcoming spe- 
cial issue oi Applied Measurement in Education devoted to the GI Forum case 
(administration of Texas high school graduation test to minority students 
upheld by a federal court). I have also recently completed invited chapters 
on Legal Issues in Standard Setting for K~12 Programs and Legal Issues 
Affecting Special Populations in Large-Scale Testing Programs to be published 
in edited volumes later this year. 

In 1993, 1 authored an assessment law handbook for policymakers enti- 
tled Legal Implications of High-Stahes Assessment: What States Should Kno’w. 
I have also published eight reviews of standardized assessments and tech- 
nical measurement texts and regularly contribute a legal issues column for 
the National Council on Measurement in Education newsletter. A full list- 
ing of my presentations and publications is provided in my vita filed in this 
proceeding. 

I have 20 years of experience working with large-scale assessments in 
more than a dozen states and several school districts, including English 
Language Learner (ELL)^ assessment issues. I have also worked with pro- 
fessional organizations and test publishers on a variety of standardized test 
instruments. In addition, I am currently a member of the Technical 
Advisory Committees for the Voluntary National Test and for the GED 
high school equivalency test. 

Prior Legal Work 

In addition to my work in California, I have served as an expert witness 
and/or consultant for cases in Arizona, Alabama, Connecticut, Indiana, 
Minnesota, Pennsylvania, Texas, and Virginia involving test disclosure, 
testing accommodations, test tampering, graduation testing of special edu- 
cation students, evaluating teachers, flagging certification test scores, test 
security and graduation testing, and teacher licensure testing. I was deposed 
and testified in the Texas graduation testing case and testified without dep- 
osition in two others: a due process hearing in Alabama and a district court 
case in Virginia.^ 

^Language minority students used to be designated as Limited English Proficient (LEP) in 
state and federal statutes. The new designation “ELL" will be used throughout this report. 

^GI Forum v. Texas Education Agency, 87 F. Supp. 667 (W.D. Tex. 2000); Golden v. 
Birmingham Board of Education 6c Alabama State Department of Education, IDEA due 
process hearing, January 1994; Pandazides v. Virginia Board of Education, 752 F. Supp. 696 
(E.D. Va. 1990), rev’d, 946 F.2d 345 (4th Cir. 1991). 
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California SBE & CDE Consultant 

I have served as a consultant to the California State Board of Education 
(SBE) on the selection of a standardized test pursuant to California Educa- 
tion Code (CEC) §§ 60600-60647 and to the California Department of 
Education (CDE) on the high school graduation test. My work as a con- 
sultant to the SBE occurred in November of 1997 and involved a psycho- 
metric evaluation of proposals submitted by test publishers for the stan- 
dardized testing component of the Standardized Testing and Reporting 
(STAR) Program, My work with the CDE includes a presentation on 
Setting Performance Standards (March 1998), attending Advisory Com- 
mittee meetings (November 1998; January 1999), a presentation on Oppor- 
tunity to Learn and Testing Accommodations (November 1999), and con- 
tinuing consultation on the high school graduation test beginning in 
September 2000. My role as a consultant to the CDE has been to provide 
technical expertise on a variety of assessment issues. 

Basis for Professional Opinions 

I have reviewed the Leroy Greene California Assessment of Academic 
Achievement Act (Assessment Act), which includes the STAR Program, 
the regulations relating to the STAR Program, the Public School Account- 
ability Act, and Proposition 221? In addition, I have reviewed numerous 
documents in the case of California Department of Education et al v, San 
Francisco Unified School District et al and San Francisco Unified School Dis- 
trict et al V. State Board of Education et al?" including pleadings, declara- 
tions, motions, exhibits, depositions, correspondence, and discovery mate- 
rials. Further, I have reviewed STAR test materials provided by the pub- 
lisher, including the Stanford Achievement Test Series, Ninth Edition 
(SAT9) Form T, designated by the SBE for administration to all California 
students in grades 2 through 11, and SAT9 test scores and Academic 
Performance Index (API) ratings from the CDE website.^ 

The professional opinions expressed in this report are based on review of 
the materials described above together with 20 years of experience in test- 
ing, extensive work with state education agencies, test publishers, and the 
federal government on all facets of large-scale assessment programs, and my 
own extensive research on testing issues. In particular, with respect to the 
SAT9, 1 supervised data analyses for the SAT7, consulted with the pub- 
lisher s staff on the renormed SAT7+, and reviewed the proposal and sup- 
porting technical materials for an evaluation of the SAT9 for the STAR 

^Cal. Educ. Code §§ 60600 et seq.; 5 Cal. Code Reg. §§ 850-874; CEC § 52050 et seq.; CEC 
§ 300 et seq. 

^San Francisco Superior Court Case No. 994049. 

Svww.cde.ca.gov. 
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program. My judgments about testing issues have been informed by exten- 
sive experience in the field and through the research and past experience of 
the profession. 

Statutory Scheme: Summary 

The San Francisco, Oakland, Hayward, and Berkeley Unified School 
Districts (SFUSD et al.)^ joined together to oppose the requirement that 
ELLs enrolled in the district for less than 30 months in grades 2 through 
11 be administered the SAT9 annually beginning in the spring of 1998. 
Provisions from several state statutes and regulations, including the Leroy 
Greene California Assessment of Academic Achievement Act, Proposition 
227, the Accountability Act, and SBE STAR regulations, when considered 
together, provide a statutory scheme for the provision and evaluation of 
public education in California and create the testing obligations challenged 
by SFUSD et al. Pertinent portions of these Education Code provisions 
and SBE regulations are summarized below. 

The Assessment Act established the STAR program and its guidelines. 
Relevant sections state: 

It is the intent of the Legislature in enacting this chapter to provide a system 
of individual assessment of pupils that has, as its primary purpose^ assisting 
pupils, their parents, and teachers to identify individual academic strengths and 
weaknesses^ in order to improve teaching and learning. It is further the intent of 
the Legislature in enacting this chapter to determine the effectiveness of school 
districts and schools^ as measured by the extent to which pupils demonstrate 
knowledge of the fundamental academic skillsy as well as the abihty to apply 
those skills. 



(a) There is hereby established the [STAR] program. 

(b) Commencing in the 1997-98 fiscal year and each fiscal year thereafter,... 

each school district^. . .shall administer to each of its pupils in grades 2 to 11, 
inclusive, before May 15, achievement test designated by the [SBE]. 

(c) [Makeup days]. 

(d) [Testing in grades 1 and 12]. 

(e) Individuals with exceptional needs who have an explicit provision in their 
individualized education program that exempts them from the testing 
requirement. ..shall be so exempt. 

(f) At the school district s option, pupils of limited EngUsh proficiency who 
are enrolled in any of grades 2 to 11, inclusive, may take a second achieve- 
ment test in their primary language. . . 

(g) Pupils of limited Enghsh proficiency who are enrolled in any of grades 2 
to 11, inclusive, shall be required to take a test in their primary language 
if such a test is available, if fewer than 12 months have elapsed after their 
initial enrollment in any public school in the state. 



‘SFUSD, OUSD, HUSD, and BUSD, hereinafter referred to collectively as SFUSD et al. 
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Based upon a review of the achievement tests submitted and the recommen- 
dation made by the Superintendent of Public Instruction..., the State Board 
of Education, in its sole discretion, based on the [psychometric, feasibility, 
cost, and experience criteria stated in § 60644], shall designate for use as part 
of the STAR Program a single test in grades 2 to 11, inclusive, no later than 
November 14, 1997. 

In designating an achievement testj [the SBE] shall adopt only a nationally 
normed test and shall consider each of the following criteria: 

(a) Ability of the publisher to produce valid^ reliable individual pupil scores. 

(b) Quality and age of empirical data supporting national norm referenced 
data analysis of the proposed assessment... 

(c) Ability to report [individual student scores, aggregated test results, and 
disaggregated scores for ELLs and nonELLs.,.] ...^ 

Further, Proposition 227 supplies a context for defining the funda- 
mental academic skills for Cauiornia school students referenced in the 
STAR legislation. In pertinent part, Proposition 227 provides: 

The English language is the national public language of the [United States 
and California], ...and is also the leading world language for science, tech- 
nology, and international business, thereby being the language of economic 
opportunity\ 

The government and the public schools of California have [a duty] to pro- 
vide all of California’s children, regardless of their ethnicity or national ori- 
gins, with the skills necessary to become productive members of our society, 
and of these skills, literacy in the English language is among the most important^ 

[Subject to parental waiver based on prior written informed consent for chil- 
dren who already know English, are 10 years or older and would benefit from 
an alternative program, or have special needs that could not be addressed dur- 
ing a one month trial period in an English language classroom], all children 
in California public schools shall be taught English by being taught in 
English. In particular, this shall require that all children be placed in English 
language classrooms. Children who are English learners shall he educated 
through sheltered English immersion during a temporary transition period not 
normally intended to exceed one year} 

Moreover, Education Code § 30, dating back to at least 1977, provides: 
English shall be the basic language of instruction in all schools. 

Bilingual instruction is authorized to the extent that it does not interfere with 
the systematic, sequential, and regular instruction of all pupils in the English 
language.’ 

Additionally, the Accountability Act implements the school evaluation 
requirement of the Assessment Act by establishing a state Academic Per- 

^CEC § 60602(a); § 60640(a)-(g); § 60642(b); § 60644(a)-(c); emphasis added. 

®CEC § 300(a)8c(c); § 310-311; § 305; emphasis added. 

’CEC § 30. 
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formance Index (API) to be calculated for each California public school, a 
state intervention program for schools performing below the national aver- 
age, and rewards for schools that meet their grov^ targets. The Accounta- 
bility Act also provides that: 

Only the test scores of pupils enrolled in a school district for one year or more may 
be included in the test results reported in the API. 

Pupil scores from the following tests, when available and when found to be 
valid and reliable for this purpose^ shall be incorporated into the API: ...(2) 
The nationally normed test as augmented... 

Finally, Title 5 of the California Code of Regulations adopted by the 
SEE provides specific rules for the administration of the STAR Program 
designated achievement test, the SAT9. In addition, recent revisions to the 
regulations adopted by the SEE make available to ELLs enrolled in a Cali- 
fornia school district less than one year^ STAR testing modifications which 
include translation of directions and use of a bilingual dictionary}^ 

Description of the SAT9 

The Stanford Achievement Test Ninth Edition (SAT9) is a nationally- 
normed, standardized achievement test administered annually to all stu- 
dents in grades 2 through 11 in California. The test measures academic 
skills in Reading, Mathematics, Language, and Spelling based on knowl- 
edge and skills commonly included in the grade level curricula of a major- 
ity of school districts in the United States. In Reading, the tested skills 
include vocabulary and comprehension; in Math, they include problem 
solving and procedures. The test items are presented in multiple-choice for- 
mat and often include pictures, graphics, or other stimulus materials. 
Except for elementary students in grades 2 and 3 who mark their answers 
in a machine-scorable test booklet, students mark their answers on a sepa- 
rate answer sheet. All test directions, questions, and stimulus materials are 
written in English. 

Professional Standards 

The primary psychometric issue raised by SFUSD et al. involves state- 
mandated testing of ELLs. In particular, SFUSD et al. question whether it 
is appropriate to administer an achievement test in English to ELLs<30 

>“CEC § 52052(a)&(b), emphasis added, 

"5 Cal. Code Reg. § 853(d) as proposed to be amended through SBE adoption on September 
7, 2000 (Office of Administrative Law pending). 

‘^Science and Social Science subtests are also administered at the high school level. Districts 
may elect to administer optional SAT9 subtests at other grade levels. 
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(students whose primary language is not English enrolled in the district for 
less than 30 months). In my professional opinion, given a state statutory 
scheme for which it is reasonable to infer an intent to hold schools 
accountable for improving the academic skills of all students, in English, 
and no negative state-imposed consequences for individual ELL stu- 
dents, administration of the SAT9 to ELLs<30 is consistent with all 
applicable professional standards. 

Relevant professional standards include the 1985 Standards for Educa- 
tional and Psychological Testing (in force at the time the SAT9 was developed 
and selected by California) and the 1999 Standards for Educational and 
Psychological Testing (a revision of the 1985 Test Standards published in late 
1999 ) ij Both the 1985 Test Standards and the revised 1999 Test Standards 
were developed and published by three national professional organizations 
whose members are involved in assessment activities: the American Educa- 
tional Research Association (AERA), the American Psychological Associa- 
tion (APA), and the National Council on Measurement in Education 
(NCME). 

The SAT9 was developed over several years in the early to mid 1990s and 
was adopted by the SBE for initial administration in California in the 
spring of 1998. The SAT9 was developed to satisfy the 1985 Test Standards 
which were in force at that time. The 1999 Test Standards were published 
after SAT9 administration in the spring of 1999. Therefore, it is my profes- 
sional opinion that the 1985 Test Standards should be applied in the present lit- 
igation. Nonetheless, the revised 1999 Test Standardsy though providing 
additional explanatory material and some extra standards for new test uses 
and nontechnical issues, have not appreciably changed the requirements 
articulated in the 1 985 Test Standards. As a result, this report will frame the 
relevant psychometric issues in the context of both sets of standards. 

Professional Judgment Affirmed 
by the 1985 Test Standards 

The STAR legislation provides that, in designating an achievement test, the 
SBE adopt only a nationaUy-normed test and consider, among other fac- 
tors, the ability of the publisher to produce valid, reliable, comparable 
scores. 

Validity and reliability are “terms of art” in the fields of psychometrics 
and educational measurement. The 1985 Test Standards stated: “The first 
task [in establishing the validity of an achievement test] is to specify ade- 

American Psychological Association (1985). AERA/APA/NCME Standards for Education- 
al and Psychological Testing, Washington, DC; American Psychological Association (1999). 
AERA/APA/NCME Standards for Educational and Psychological Testing, Washington, 
DC [hereinafter 1985 Test Standards and 1999 Test Standards], 

"CEC § 60644. 
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quately the universe of content that a test is intended to represent, given the 
proposed use of the test” (p. 10), That is, test validity must be evaluated 
based upon expert judgment of the relationship between the domain of 
content to be tested and the stated purpose for testing. Moreover, a com- 
ment to Standard 13.1 in the 1985 Test Standards, which deals with the 
validity and reliability of testing non-native English speakers, advises that 
“careful professional judgment is required to determine when language dif- 
ferences are relevant” (p. 74). 

Professional Judgment Reaffirmed 
by the 1999 Test Standards 

In corresponding text from Standard 9.1 of the 1999 Test Standards dealing 
with reliability and validity of test score interpretation for ELLs, the com- 
ment states: “Assessment methods together with careful professional judg- 
ment are required to determine when language differences are relevant. Test 
users can judge how best to address this standard in a particular testing sit- 
uation” (p. 97). A caution in the Introduction to the 1999 Test Standards 
states: “When tests are at issue in... venues requiring expert witness testi- 
mony it is essential that professional judgment be based on the accepted 
corpus of knowledge in determining the relevance of particular standards in 
a given situation. The intent of the Standards is to offer guidance for such 
judgments” (p. 4). 

The above quotes from the 1985 Test Standards and the 1999 Test 
Standards underscore the importance of psychometric training and expert- 
ise for those who offer opinions about the guidelines contained in the Test 
Standards, specific Standards on validity and reliability, and the application 
of the Test Standards in specific testing contexts. 

Validity 

Validity refers to the weight of accumulated evidence supporting a particular use 
of test scores, SAT9 test scores are used by the state to determine whether 
schools are meeting their growth targets for academic improvement and by 
schools and parents to identify individual students’ strengths and weak- 
nesses, The most important evidence of validity in this situation is a meas- 
ure of the degree to which the items on each subject matter test measure 
the knowledge and skills identified by the State as important for aU stu- 
dents to achieve. This type of validity evidence is referred to as content 
validity evidence. 

Content Validity Evidence 

Standards 1,6-1. 7, 3.3, and 3.5 from the 79^5 Test Standards, and Standards 
1.6, 1.7, 3.2-3. 3, 3.5-3. 9, 3.11, and 13.3 from the 1999 Test Standards deal 
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specifically with issues related to content validity evidence. These Standards 
require that the purpose of the test, procedures used to specify the content 
domain, the qualifications of content experts, and the procedures used to 
obtain expert judgments be clearly documented. These requirements for 
developing content validity evidence are described more fully below. 

As indicated in the Test Standardsy content validity evidence for an 
achievement test is typically obtained by professional judgment. Based on the 
purpose of the test, a diverse panel of content experts is asked to define an 
appropriate domain of academic subject matter, to develop a set of test 
specifications which identifies the specific knowledge and skills to be sam- 
pled from the domain, and to specify the proportional weight to be given to 
each sampled content area. 

After trained item writers have produced a set of items based on the test 
specifications, the diverse panel of content experts is asked to review each 
potential test item and determine whether it measures the intended subject 
matter skill. As part of this review, these content experts also check the cor- 
rectness of the keyed answer, check for ambiguities in wording and other 
potential item flaws, evaluate the appropriateness of the content and diffi- 
culty of the item for the intended grade level, and identify any inappropri- 
ate or potentially offensive language or content that might impair accurate 
assessment of minority students such as African-Americans or Hispanics. 

The edited items are then field tested on a sample of students, item sta- 
tistics are calculated, and the items are evaluated again. During this second 
review of items, content experts re-examine the match of the item to the 
skill it is supposed to measure in the context of item data from the field test, 
including consideration of differential performance by ethnic groups such 
as African-Americans and Hispanics. Test forms are constructed based on 
the content specifications of the test and are then administered to repre- 
sentative national samples of students to develop the test norms. 

The SAT9 was developed using the extensive test development proce- 
dures described above. In addition, when the SBE considered the publish- 
er’s proposal for adoption of the SAT9, the SBE was furnished with docu- 
ments that matched the SAT9 content to the California state standards in 
each subject area. The quality and publisher’s documentation of the SAT9 
test development effort, the information on the match of the SAT9 to the 
State Content Standards, and evaluations by the SBE’s expert panel pro- 
vided the SBE with the necessary information to judge the content validi- 
ty of the SAT9. 

The SBE’s evaluation panel rated each publisher’s test proposed for the 
STAR Program on a variety of factors. In particular, panel members were 
asked to rate each proposal on each of the statutory criteria including the 
“ability to produce valid, reliable, individual pupil scores” [§ 60644(a)] and 
the “quality and age of empirical data supporting national norm-referenced 
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data analysis of the proposed assessment” [§ 60644(b)]. Panelists were also 
asked to provide comments on strengths and weaknesses relative to each 
criterion and to provide additional comments to assist the SEE in evaluat- 
ing the proposals. Based on extensive information provided by the publish- 
er, the SEE's evaluation panel (of which I was a member) judged the SAT9 
to be valid for the assessment uses described in the STAR statute. 

Purpose for Testing 

The STAR legislation quoted earlier states the legislative purpose of the 
STAR tests to be, primarily, the measurement of individual students' 
strengths and weaknesses in fundamental academic skills, and secondarily, 
the evaluation of school effectiveness based on aggregation of the individ- 
ual student assessment results. The Legislature directed the SEE, with 
input from expert consultants who considered, among other factors, the 
validity and reliability of the proposed tests, to adopt a single, nationally- 
normed, standardized achievement test to be administered annually to all 
students in grades 2 through 11 statewide beginning in the spring of 1998. 
In November 1997, the SEE adopted the SAT9 for this purpose, 

SFUSD et al. argue that ELLs should not be tested with the SAT9 until 
they have achieved adequate proficiency in English. They maintain that the 
test should measure academic skills independent of the student's language 
proficiency in English. They have proposed a policy in which ELLs are 
tested only after having been enrolled in the district for 30 months. SFUSD 
has implemented this policy by exempting all ELLs enrolled in the district 
for less than 30 months (ELLs<30) from SAT9 testing unless their teach- 
ers recommend that they be tested.^^ In 1999, OUSD used teacher judg- 
ments of language proficiency to exempt some Category 1 ELLs from test- 
ing.^* HUSD and BUSD have tested their ELL students but object to the 
state requirement to do so.^^ 

Contrary to SFUSD et al.’s assertions, sound measurement theory does 
not require that subject matter content be tested separately from language 
proficiency. Sound measurement theory specifies that educational tests 
should measure what students are expected to learn and what the test 
administrator (in this case, the state) wants to evaluate. The introductory 
text to the Validity chapter in the 1999 Test Standards states: “In education- 
al program evaluations, ...tests may properly cover material that receives 
little or no attention in the curriculum, as well as that toward which instruc- 
tion is directed” (p. 12). The school API measures derived from SAT9 test 



'^Deposition of Jim Stack, May 4, 2000, p. 169: 24—170:3; 213:19—214:9. 

'^Deposition of Alma Williams, August 9, 2000, p. 340:10-17. 

'^HUSDs Supplemental Responses to SBE’s Amended Second Set of Special Interrogatories, 
No. 10; BUSD’s Second Response to SBE’s Fourth Set of Special Interrogatories, No. 36. 
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scores are an example of educational program evaluation referred to in the 
1999 Test Standards. 

Assessment of Academic Skills in English 

Both the 1985 Test Standards and the 1999 Test Standards indicate that 
making a professional judgment about the validity and appropriateness of 
the SAT9 for ELLs requires consideration of the purpose for testing. The 
STAR legislation^® specifically required the SBE to select a single, nation- 
ally-normed achievement test for administration to all students in grades 2 
through 11 in California. The only exceptions were special education stu- 
dents, whose individualized education programs (lEPs) exempted them 
from testing, and students whose parents sign a waiver requesting that their 
children not be tested.^’ At the time the legislation was enacted, as today, 
all nationally-normed achievement tests in use in the United States test 
academic skills in English. 

In addition, the STAR legislation made specific provision for mandato- 
ry primary language testing for ELLs enrolled in a school district for less 
than 12 months (ELLs<12), if available, with a district option to test other 
ELLs. Thus, the STAR statute made specific reference to additional testing 
of certain ELLs but did not include them when specifying exclusions from 
the main nationally-normed test administered to all students. Therefore, it 
is reasonable to conclude that the Legislature intended that all ELLs’ 
academic skills be tested in English. The SAT9 selected by the SBE and 
implemented by the CDE is designed to provide a global evaluation of the 
achievement of all students in specified academic subjects in English. 

Moreover, the Public School Accountability Act (Accountability Act) 
states that “the purpose of the California public school system is to provide 
for the academic development of each pupil and prepare each pupil, to the 
extent of his or her ability, to become a lifelong learner, equipped to live and 
succeed within the economic and social complexities of the 21st century.”^® 
In a country where the language of post-secondary education, commerce, 
and the workplace is English, and in a state in which the law requires aU 
students to be taught in English, it is appropriate to measure student 
achievement in English. 

When the purpose of a test is to measure academic achievement in 
English, there is no confounding when students with either limited con- 
tent knowledge or limited English proficiency receive low scores on the 
test. English proficiency is not confounding the measurement of subject 
matter achievement, it is an integral part of it. 



*'CEC § 60600 et seq. 

•’CEC § 60640(e); Cal. Code of Reg. § 852(a). 
”CEC § 52050.5. 
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A low score on the test obtained by an ELL or a low- achieving native 
English speaker is a valid indication that the student is not yet able to 
demonstrate the expected academic achievement in English. Psychometri- 
caUy, the fact that ELLs may be able to demonstrate some of the tested 
skills in another language is no substitute for the intended measurement of 
those skills in English. Being able to demonstrate skills in English and 
demonstrating those same skills in another language are not equivalent. 
To measure ELLs* progress in learning academic skills in English, those 
students must be assessed in English. 

School Accountability for All Students 

The Accountability Act provides for assistance to underperforming schools 
and rewards for successful schools based on an Academic Performance 
Index (API) which currently includes the only available valid and reliable 
state^vide measure, SAT9 scores. Each school will be evaluated on 
improvement based on its own baseline using a minimum annual growth 
target percentage for each numerically significant ethnic and socioeconom- 
ic subgroup.^^ Scores for students who have been enrolled in the district for 
less than 12 months, including ELLs, are not included in the API.“ 
Schools in SFUSD, OUSD, HUSD, and BUSD have voluntarily elected to 
submit proposals for state funding for low-performing schools under the 
Accountability Act’s II/USP program.^ A study presented to the Account- 
ability Advisory Committee indicated that schools with a higher propor- 
tion of ELLs had a slightly higher projected growth on the API than 
schools with a lower proportion of ELLs.^'* 

There is no viable alternative to a nationally-normed achievement test 
currently available for achieving the purpose of measuring each student’s 
content area achievement in English and holding schools accountable for 
growth and improvement of the tested skills for all low-scoring students, 
including ELLs. Presumptively excluding ELLs enrolled in school for less 
than 30 months from SAT9 testing prevents the State from gathering as- 
sessment information on an entire category of students. Any student with 
extremely weak academic and English language proficiency is at high risk 
of dropping out of school, failing to earn a high school diploma, being 
unable to meet criteria for admission to post- secondary education, or being 
forced to work in an unskilled job with low pay and no opportunity for 
advancement. Early and sustained attention to academic skill development 
and English language proficiency affords every student an opportunity to 

'^CEC § 52052(a)8c(c); § 52057. 

“CEC § 52052(a)8c(c). 

”CEC §§ 52053-52054.4. 

^^Deposition of William Padia, May 2, 2000, p. 79. 
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benefit from available economic opportunities. 

Waiting until an ELUs fourth year of public school enrollment to administer 
the SAT9 as advocated by SFUSD et al voould result in no statewide accounta- 
bility of these students' schools for their progress in learning skills in English for 
the first three years of instruction. Waiting four years to initially test ELLs<30 
would also deprive parents and students of a source of information identi- 
fying academic strengths and weaknesses, including a possible need for 
instructional assistance. A test in the primary language, when available, may 
provide additional information about a student's academic achievement. 
However, if the ultimate purpose for testing is to ensure that all students 
master academic skills in English in a timely fashion, a primary language 
test alone is not sufficient. 

Measuring Academic Skills Independent of English Proficiency 
The declarations of SFUSD et al. s experts that the SAT9 test is invalid for 
ELLs are based on an assumption that the purpose for testing is to meas- 
ure only content knowledge and skills independent of English proficiency. 
This purpose asserted by SFUSD for testing is different from the. rea- 
sonable inference of testing academic skills in English derived from the 
state statutory scheme described earlier. SFUSD et al.’s purpose, howev- 
er, can be achieved by administering a second achievement test to ELLs in 
their primary language, an option provided to school districts under 
§60640(f) of the CEC and required, if available, for ELLs enrolled in 
school for less than 12 months.^^ The SBE has designated the Spanish 
Assessment of Basic Education, Second Edition (SABE2) as the primary 
language test for Spanish-speaking students, who comprise about 82 per- 
cent of the California ELL population No other tests \vith adequate 
validity and reliability are currently available for assessing academic skills in 
other primary languages. 

Administering an achievement test in a student's primary language may 
assist schools in evaluating whether low SAT9 test scores are due to lack of 
content knowledge and skills, lack of English proficiency, or both. How- 
ever, just as low scores on tests administered in English may be due to poor 
English skills, low scores on tests administered in a student's primary lan- 
guage may be due to lack of proficiency in the primary language. Thus, dis- 
tricts must also interpret low scores on primary language tests with cau- 
tion. 

Reasons for Poor Test Performance 

The SAT9 validly determines whether an ELL or native English speak- 



“CEC § 60640(g). 

“CDE Statistical Report, 1999, p. x. 



PHUIPS 

f 13 



er can demonstrate the requisite academic skills in English; the state 
need not establish nor is the SAT9 designed to determine \^y a low-per- 
forming student scores poorly. In general, standardized achievement tests 
do not provide explanations for low test scores for any students, and there 
is no psychometric standard requiring that they do so. That is the job of 
local educators who are responsible for identifying effective instructional 
interventions to improve the skills of low-scoring students. 

There are many factors that may negatively affect a student’s achieve- 
ment of academic skills. They include, but are not limited to, economic dis- 
advantage, lack of parental involvement, poor attendance, low motivation, 
lack of appropriate instruction, substance abuse, personal problems, illiter- 
acy, and limited English proficiency. These factors do not invalidate the test 
results for either ELLs or nonELLs, but their identification can aid school 
staff in understanding why a student has not achieved so effective instruc- 
tional assistance can be provided. Such assistance could include language 
proficiency instruction, content area instruction, counseling, tutoring, par- 
ent conferences, or a combination of these or other support services. 

Assessment of Integrated Skills 

Like the combination of English language proficiency and content achieve- 
ment assessed by the SAT9, other statewide tests also involve the assess- 
ment of integrated skills that work together to produce the desired achieve- 
ment outcome. For example, on a science test, students might be asked to 
write an evaluative essay about two scientific reading selections with oppos- 
ing points of view. Achieving a high score on such a task requires the inte- 
gration of reading and writing skills with the relevant content knowledge 
and skills in science. Poor readers and writers may achieve lower scores even 
if they know the tested science content. These low scores are considered 
valid when state content standards indicate that reading and writing about 
science content is considered an important part of the science achievement 
being measured. 

If the SFUSD et al.’s point of view advocating isolated assessment of 
academic content were adopted here, such science tasks would be judged 
invalid for native English speakers because they measure more than pure 
science knowledge. But many educators would find a “pure science” assess- 
ment (wdth minimal reading and no viriting) inconsistent with the content 
standards, curricula, and real world applications expected of students. Thus, 
in this context it would be inappropriate to allow a district to narrow the 
intended science achievement by exempting all its students who were poor 
readers or writers from a science test that includes those skills. 

Another example of testing integrated skills is the two-step story prob- 
lem found in many mathematics tests. In order to answer such a question 
correcdy, the student must be able to do both steps in the problem. Lack of 
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knowledge of either step required by the problem will result in an incorrect 
answer. The fact that the test score alone does not indicate whether incor- 
rect answers are due to the inability to do step 1 or step 2 does not mean 
that the test should include only 1-step story problems. The ability to com- 
plete both steps is necessary to demonstrate the skill of solving realistic 
word problems. It is the responsibility of local educators to collect addi- 
tional data to determine which step the student cannot do and then to 
design appropriate instruction to teach the missing skill to the student. 

Similarly, it would be inappropriate for a district to exempt students, 
such as ELLs<30, for whom English may be challenging, from a statewide 
assessment intended to measure content knowledge in English for all stu- 
dents. If this argument were adopted, to be fair to everyone, native English 
speakers who have impaired English proficiency should also be exempted 
from testing because they too are unable to demonstrate their true content 
knowledge due to poor English skills. The result of adopting such a policy 
’would be to exempt from identification the very students (and schools with many 
such students) who most need additional educational assistance. 

Comparable Data 

The STAR Program statute specifies that the nationally-normed achieve- 
ment test adopted by the SEE provide comparable scores.^^ To obtain com- 
parable scores, the same test must be administered to all students under the 
same standardized administration conditions as those employed when the 
national norm group was tested. If some students were to take a different 
test than others, or if the test administration conditions differed across stu- 
dents (e.g., time limits differed or the test was read aloud to some students), 
comparisons of the obtained scores to the national norms would not be 
valid because the students tested under different conditions would not be 
demonstrating achievement of the same skills. 

Similarly, including scores that were not comparable would produce 
invalid aggregate results for tracking the progress of schools and subgroups 
of students within schools over time. Further, without the normative infor- 
mation from the test, parents would be unable to compare their children’s 
progress with that of other students in the nation or their children’s school 
with that of other schools in California on a national standard. 

Normative Comparisons 

By design, achievement “at grade level” on nationally standardized tests in 
the United States has been defined by average student performance on 
grade-level appropriate academic content tested in English. This baseline 
national performance provides a benchmark for comparison of a student’s 



^CEC § 60643(a)(5)(F). 
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test performance with that of students at the same grade level nationally. 
Thus, requiring that the selected test be normed on ELLs would defeat the 
purpose of administering a nationally-normed test. Separate ELL norms 
would only indicate how an ELLs achievement compared to other ELLs 
but would not indicate whether an ELL had achieved what was expected at 
the student’s grade level. That is, an ELL could score well in comparison to 
ELL norms but poorly in comparison to national norms. 

Introductory information in Chapter 9, Testing Individuals of Diverse 
Linguistic Backgrounds, in the iPPP Test Standards states: “test norms based 
on native speakers of English either should not be used with individuals 
whose first language is not English or such individuals’ test results should 
be interpreted as reflecting in part current level of English proficiency 
rather than ability, potential, aptitude ...” (p. 91, emphasis added). 

It is clearly understood that the SAT9 measures achievement at a partic- 
ular point in time, not ability or aptitude. Moreover, SAT9 scores can be 
interpreted as measuring both academic content and English proficiency, 
one of the alternatives the statement quoted above from the 1999 Test 
Standards indicates is appropriate. As such, use of SAT9 normative scores 
for ELLs is appropriate if the state’s purpose is to measure current academic 
skills in English and to hold schools accountable for expected improve- 
ments in those skills over time. Furthermore, receipt of additional educa- 
tional assistance to remediate language and skill weaknesses is a positive 
outcome of testing and a benefit that the accountability system will encour- 
age schools to provide to all low-performing students, including ELLs. 

Descriptive information in Chapter 9 of the 1999 Test Standards that 
addresses differences in test performance across linguistic groups due to dif- 
ferences in language proficiency, acculturation, vocabulary, educational 
background, test-taking skills, etc. is equally applicable to native English 
speakers whose test performance may also be adversely affected by any of 
those factors. Standard 9.3 states: “The test generally should be adminis- 
tered in the test taker’s most proficient language, unless proficiency in the 
less proficient language is part of the assessment” (p. 98, emphasis add- 
ed). From the statutory scheme described earlier which mandates instruc- 
tion in English for all students, it is reasonable to conclude that proficien- 
cy in English, a language which may be the language of least proficiency for 
ELLs initially and a serious weakness for some native English speakers, is 
an intended part of the SAT9 assessment for all students because it is an intend- 
ed part of the curriculum for all students. 

Instruction in English for All ELLs 

Proposition 227, English Language for Immigrant Children, requires that 
ELLs be taught English and academic subjects in English beginning in the 
fall of 1998.^® After a temporary transition period of approximately one year 
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in an English Immersion Program, ELLs are to be transferred to regular 
classrooms where instruction is provided in English. Parents can sign a 
waiver requesting that their children be placed in an alternative program 
which may include a bilingual program.” 

Bilingual programs have been generally understood to provide instruc- 
tion in the student’s primary language and English. The sunsetted 
Bilingual-Bicultural Education Act of 1976 provided: 

The Legislature finds and declares that the primary goal of all programs 
under this article is, as effectively and efficiently as possible, to develop in 
each .child fluency in English. 

As the pupil develops English language skills, the amount of instruction 
offered through English shall Increase. 

“Bilingual-bicultural education” is a system of instruction which uses two 
languages, one of which is English, as a means of instruction.^® 

The SABE2 provides a measure of academic progress in Spanish for 
ELLs whose primary language is Spanish, but there is no corresponding 
test of academic progress in any of the more than 50 primary languages rep- 
resented in the state.^^ Therefore, if assessment were confined only to"* that 
part of an ELL’s academic learning that occurs in the native language, there 
would be two negative consequences: (1) only Spanish bilingual students 
would be assessed — all other ELLs would be excluded; and (2) no system- 
atic, comparable information would be available to evaluate ELLs’ progress 
in learning academic skills in English even though aU should be receiving 
at least a portion of their instruction in English. Standardized achievement 
tests provide an important benchmark for tracking student progress and 
ELLs would be disadvantaged by being excluded from this important tool 
for measuring progress. 

Inclusion of ELLs in National Norms 

The SAT9 standardization sample was selected and statistical weighting 
procedures applied so that the final norming group closely resembled the 
demographic characteristics of students nationally on the characteristics of 
geographic location, socioeconomic status, urbanicity, and ethnicity from 
Census data. At the rime, no data were available on enrollment of ELL stu- 
dents nationally. 

Districts agreeing to participate in the norming sample were given the 
following policy directive for deciding which students to test: “Students 
receiving instruction as part of a regular education classroom who would 

“CEC § 300 et seq. 

^CEC §§310-311. 

“CEC § 52161; § 52163(a)(2)&(b) (sunsetted June 30, 1987). 

*'See CDE Language Census Summary Statistics, 1998-99. 
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normally test with other students in the regular classroom were asked to be 
part of the standardization sample.”^^ At the time, few ELL students in U.S. 
schools met this criteria for testing. As a result, only about 2 percent of the 
students in the spring norm group were ELLs.^^ With the passage of 
Proposition 227 in California and measures in other states leading to 
greater inclusion of ELLs in regular education classrooms and statewide 
testing programs, the next time the SAT9 is revised and renormed, districts 
should be more willing and able to include ELLs. 

Further, if California wished to do so, it could create California norms 
for the SAT9 whose demographic representation would match that of stu- 
dents in the state. However, such norms would have limited usefulness 
because they would be state rather than national norms. Such state norms 
would be unlikely to mirror national performance because the California 
norm group would contain proportionately more ELLs than the national 
student population, even if the national norm group included ELLs in pro- 
portion to their representation nationally. To the extent that ELL perform- 
ance is lower than the national norm group, California state norms might 
mislead users into believing a student was performing well when compari- 
son to the national norm group would indicate a much weaker perform- 
ance. 

Other Types of Validity Evidence 

Other types of validity evidence include criterion and construct validity evi- 
dence. Criterion validity evidence, in the form of correlation coefficients, is 
most appropriate for situations in which test scores are used to predict out- 
comes, such as freshman grade point averages predicted from college 
admissions test scores. Criterion validity evidence can also be useful in 
determining the degree to which two tests measure the same or different 
skills. Because the SAT9 tests are intended to measure specified content 
knowledge and skills, and not to predict any other outcome or replace 
another test instrument, criterion validity evidence is not necessary or 
appropriate for establishing the validity of the SAT9 for use in the 
California STAR program. 

Construct validity evidence refers to the sum of research knowledge and 
experiments designed to define a psychological construct, such as extrover- 
sion or depression, that an instrument is intended to measure. Because the 
SAT9 tests are achievement tests designed to direcdy measure specific aca- 
demic content, not to define unobservable psychological constructs, con- 
struct validity evidence is also not required for establishing the validity of 
the SAT9 for use in the California STAR program. However, to the extent 

^^SAT9 Technical Manual, p. 27. 

”SAT9 Technical Manual, p. 28, 
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that the SAT9 subtests measure general educational constructs such as 
reading or math, evidence that an item administered below its targeted 
grade level appears harder and that same item administered above its tar- 
geted grade level appears easier provides some evidence that the SAT9 
measures a sequenced educational curriculum for which items are placed at 
the appropriate grade level. 

Validity Evidence for ELLs 

There is no requirement in the 1985 Test Standards or the STAR statute 
that the selected standardized achievement test be validated or normed on 
disaggregated student subgroups such as ELLs<30. However, both the 
1985 Test Standards (Standard 13.3) and the 1999 Test Standards (Standards 
9.2 and 11.22) expect publishers to provide detailed documentation by 
which users can judge the appropriateness of a specific use of a specific test 
with linguistically diverse test takers. Such information was available to the 
SEE at the time the SAT9 was selected. 

Standards 9.2 and 11.22 of the 1999 Test Standards do not address 
ELLs<30 specifically but do recommend parallel investigation of the .valid- 
ity of test score interpretations for ELLs. For example, collection of sepa- 
rate validity evidence might be appropriate in a predictive validity situation. 
However, the introductory text and comments to these standards indicate 
that the recommendation for separate collection of validity evidence for 
ELLs is premised on the assumption that the test user does not intend 
English language proficiency to be part of the tested content. In contrast, 
as already indicated, one can reasonably infer otherwise from the statutory 
scheme in California. Thus, Standards 9.2 and 11.22 are not directiy appli- 
cable to the STAR Program, Nevertheless, the publisher does report disag- 
gregated results for ELLs as required by statute so that users can evaluate 
the performance of ELLs separately. 

Summary of Validity Evidence 

The SAT9 is an achievement test that measures students' acquisition of 
specified skills at a particular point in time. The skills measured by the SAT9 
are sensitive to instruction^ and students' proficiencies are expected to improve 
over time. The STAR legislation requires specific information to be collect- 
ed and reported but imposes no negative consequences on any individual stu- 
dents, including low-scoring ELL students. 

Thus, the relevant evidence for evaluating the validity of the SAT9 for 
the purpose of measuring academic skills in English is content validity 
evidence. The extensive content validity evidence for the SAT9 meets all 
the relevant professional standards described earlier. Applying this con- 
clusion to ELLs<30 in particular, the SAT9 test is valid for ELLs<30 
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because it measures the academic content the SBE has determined all stu- 
dents should know and does so in the English language, which can be rea- 
sonably inferred from the statutory scheme to be the intended language of 
instruction and assessment for all students, including ELLs<30 and 
ELLs>30. 

Opportunity to Learn 

SFUSD et al. argue that ELLs with less than 30 months of public school 
instruction should not be tested in English because they have not yet had 
an opportunity to learn enough English. This “opportunity to learn” argu- 
ment has been borrowed from the context of statewide testing with high- 
stakes consequences for individual students (e.g., denial of a high school 
diploma). When high school students are required to pass a state gradua- 
tion test to receive a high school diploma, professional standards and legal 
cases agree that in fairness to all students, the state must demonstrate that 
its schools are teaching the tested content. 

The relevant standard from the 1985 Test Standards is Standard 8.7, 
which states: 

When a test is used to make decisions about student promotion or graduationy 
there should be evidence that the test covers only the specific or generalized 
knowledge, skills, and abilities that students have had the opportunity to 
learn. (P. 53, emphasis added.)^^ 

However, the STAR Program is distinguishable from a high school 
graduation or grade promotion test. For all students, including ELLs, the 
SAT9 is equivalent to a no-stakes pretest which is given to obtain informa- 
tion about what the students already know and to inform them of what they 
Avill be learning in the future. The state imposes no negative consequences 
on individual students and each school district has complete control of how 
individual SAT9 scores are used. The misuses of test scores cited by SFUSD et 
al as harmful to ELLs are not inevitable and are under SFUSD et al i control 
SFUSD et al. have control over score interpretations offered to students 
and parents and any use of test scores in making placement decisions for 
individual students. It is within SFUSD et al.’s power to tell ELL students 
and their parents that the test measures skills these students have not yet 
been taught and to explain when and how these students will receive the 
requisite instruction. 

It has been my experience that schools routinely test students in subject 
matters in which the student has not yet achieved proficiency. For example, 
classroom teachers often give pretests prior to beginning a new unit in order 
to find out what students already know about the content to be studied. The 

^niie corresponding standard, with similar wording, from the 1999 Test Standards is Stand- 
ard 13.5. 
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teacher can then tailor instruction based on the students’ strengths and 
weaknesses. The Assessment Act specifies a similar remedial purpose of 
“assisting pupils, their parents, and teachers to identify individual academ- 
ic strengths and weaknesses, in order to improve teaching and learning.”^^ 

If students could only be tested on skills they had already mastered, edu- 
cators and policymakers would have to make decisions without data. This 
would produce the absurd result of not knowing the student’s achievement 
but being unable to test for it because it is unfair to ask students questions 
they do not already know. Moreover, if students had to be tested only on 
content they already knew, the state would be unable to challenge districts 
and schools to meet higher standards. 

Particularly when there are no stakes for individual students, like SAT9 
testing of ELLs, having more information for decisionmaking is better than 
having less. As indicated in a later section, some districts have successfully 
accelerated the progress of their ELL students and have demonstrated that 
these students can learn the academic skills in English tested by the SAT9 
in a relatively short time. 

Teaching the Tested Skills 

It is clear from the experience of other states that what is tested is what 
gets taught, SFUSD et al. decry “teaching to the test,” and they are correct 
if they are referring to inappropriate test preparation that teaches specific 
test items. But if teachers engage in appropriate teaching of the tested skills, 
it win improve the achievement of all students on the academic skills meas- 
ured by the test. On the other hand, if ELLs are excluded from SAT9 test- 
ing, it will provide an incentive for their schools to ignore their academic 
progress and to shift resources to other students whose scores “count” 
toward the schools’ accountability ratings. 



effects, in addition to those of the 
test scores themselves, which are attributable to a testing program. Such 
indirect effects may be beneficial or detrimental. Standard 1.23 in the 1999 
Test Standards states: 

When a test use or score interpretation is recommended on the grounds that 
the test or the testing program per se will result in some indirect benefit in 
addition to the utility of information from the test scores themselves, the 
rationale for anticipating the indirect benefit should be made explicit. Logical 
or theoretical arguments and empirical evidence for the indirect benefit 
should be provided. Due weight should be given to any contradictory find- 
ings in the scientific literature, including findings suggesting important indi- 
rect outcomes other than those predicted. 



Consequential Validity 

Consequential validity refers to indirect 



i 
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Comment: For example, certain educational testing programs have been advo- 
cated on the grounds that they would have a salutary influence on classroom 
instructional practices or would clarify students' understanding of the kind or 
level of achievement they were expected to attain. To the extent that such 
claims enter into the justification for a testing program, they become part of 
the validity argument for test use and so should be examined as part of the 
validation effort. (P. 23.) 

Some of the issues identified in this Standard have already been addressed 
in earlier sections. Additional consequential validity evidence for adminis- 
tration of the SAT9 test to ELLs<30 is discussed in the sections that fol- 
low. 



Access to Instructional Help and Resources 

For both native English speakers and ELLs, it is useful to administer the 
SAT9 to communicate statewide expectations, to determine general levels 
of academic functioning in English, and to track improvement over time. 
Identifying students with low performance on state goals is a necessary first 
step in providing additional instructional help and resources where they are 
most needed to teach and reinforce the skills that have not yet been mas- 
tered. 

Tests Are Not Instructional Programs 

SFUSD et al. have argued that testing ELLs with the SAT9 in English 
denies access to instructional programs in violation of the Equal Educa- 
tional Opportunities Act.^^ However, the SAT9 itself is not an “instruc- 
tional program” as that term is typically used. Contrary to the assertion of 
SFUSD et al.’s experts (p. 7 ftn 3), students do not improve their skills by 
repeatedly taking the SAT9, but rather, by the targeted instruction that 
should occur between annual administrations of the SAT9. 

SAT9 test results can assist a local school district in monitoring whether 
all groups have access to educational programs that provide an opportunity 
to achieve educational goals. An ELL <30 who demonstrates little pro- 
gress in learning content area skills in English over several years may not 
be receiving equal access to educational experiences targeted toward 
state educational standards. Exempting ELLs<30 from SAT9 testing or 
testing ELLs<30 only in their primary language deprives educators and 
parents of an early warning if ELLs<30’s content skills in English are not 
increasing each year. Waiting four years to identify students who need addi- 
tional help may be too late to reverse ELLs<30’s content deficiencies in 
English sufficiently for them to compete favorably in the future with their 
nonELL peers. 



“20 U.S.C. §1703. 
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Familiarity with Standardized Tests 

In order to compete equally with their peers, ELLs<30 need familiarity 
with testing formats, administration conditions, item types, and test taking 
strategies in English. Administration of the SAT9 test to ELLs<30 every 
year will provide opportunities for these students to become familiar with 
the tests and learn what is expected of them. These students will see visible 
evidence of their progress in learning content in English as their scores on 
the SAT9 test increase each year. In addition, the SAT9 test will begin 
familiarizing them with the standardized testing process. This familiarity 
should help them when they reach the point of taking reclassification tests, 
placement tests, high school graduation tests, college entrance exams, or 
other significant tests given in English during their educational careers. 

School Accountability 

The only accountability that has been imposed by the state in conjunction 
with SAT9 testing under the STAR Program is at the school level. The 
Accountability Act provides incentives for schools to find more effective 
means to help all students, including ELLs<30, to increase their achieve- 
ment of state educational goals over time. This outcome will benefit 
ELLs<30 by better preparing them to compete with their native English- 
speaking peers for desirable school programs, post-secondary education, 
and employment. 

Thus, in presumptively preventing ELLs<30 from taking the SAT9, 
SFUSD et al. may not be acting in ELLs<30’s long-term best interests and 
may be depriving them of an important educational experience. For exam- 
ple, a local newspaper reported that “The [SFUSD’s] belief that testing 
children who speak little English is pointless... [and] has cost the district 
$640,000 in new state funds aimed at educating those very students.”^^ 

Allegations of Harm to ELLs 

SFUSD et al. have asserted that ELLs<30 will be harmed by the adminis- 
tration of the SAT9 test in English because their low scores wiU be stigma- 
tizing, will diminish their self-esteem, and will cause them to be inappro- 
priately placed in special education programs and portrayed as having infe- 
rior employment sldlls.^® Such misuses of test scores can occur for any stu- 
dent, including ELLs>30 and nonELLs, and underscore the importance of 
the school district’s role in accurate interpretation and use of the SAT9 test 
scores for all students. SAT9 scores provide a snapshot of achievement at a 
point in time and do not represent fixed aptitudes or employment abilities. 
In addition, individual student test scores are confidential and should not 
be available to any outside entity that might inappropriately stigmatize a 

^^Guthrie, J. Test policy costs S.F. schools, The San Francisco Examiner, April 20, 2000. 
*®SFUSD Complaint 43-46. 
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student.” 

The state has used SAT9 test scores appropriately to evaluate schools. 
The state has not engaged in any of the inappropriate uses of SAT9 scores 
cited by SFUSD et al. because the state neither imposes negative conse- 
quences on students, nor deprives them of any educational or other oppor- 
tunities, nor prescribes specific courses of study based on SAT9 test scores. 
Thus, it is up to the local districts to ensure that SAT9 testing information 
is used responsibly. SFUSD et al. have the authority to determine how the 
test scores will be used locally and an affirmative duty to communicate its 
intended use to parents as provided in § 60641(b).'" School districts can 
adopt policies that prevent misuse of ELLs<30s test scores and provide for 
the collection and use of supplementary data when making placement and 
instructional decisions for ELLs<30. Indeed, the state encourages districts 
to use multiple sources of information, not a single test score, when making 
educational and placement decisions for all students, including ELLs<30, 
in conformance with Standard 8.12 of the 1999 Test Standards and 
Standard 13.7 of the 1999 Test Standards."'^ When evaluating ELLs for re- 
tention or promotion, SFUSD et al. have stated that they do not use SAT9 
scores. 

SFUSD et al.’s experts have also alleged that the experience of taking the 
test in English will cause harm through test anxiety manifested by physical 
illness. ELLs<30 should be no more frustrated or anxious than ELLs>30 or 
nonELLs with extremely poor English language skills. Test anxiety is not 
limited to low performing students — any student can become so concerned 
about taking a test that it interferes with performance on the test. Studies 
have shown performance declines for students with both too little and too 
much test anxiety. Scores are maximized when students are concerned 
enough to work hard learning the tested skills and to do their best when an- 
swering the test questions. As one leading measurement text has observed: 

Anxiety is a frequent side effect of testing, whether that testing occurs in the 
classroom, on the athletic field, in the art exhibit hall, in the courtroom, in 
the conference room where a crucial business decision is being discussed, or 

^®CEC § 60607(d). “Individual pupil test results may only be released with the permission of 
the pupil's parent or guardian.” CEC § 60641(c). When reporting aggregate results, the oper- 
ative rule is that if the number of students in any reporting category is less than 10, no aggre- 
gate results are reported. This policy is consistent with CEC § 60641(d), which specifies that 
the score or relative position of any individually ascertainable pupil not be reported. 

^CEC § 60641(b) states in part: “The written report shall include a clear explanation of the 
purpose of the test, the pupil’s score, and its intended use by the school district.” 

^'CEC § 48070.5; CEC § 60602(a)(1) (“The Legislature recognizes that, in addition to the 
statewide assessments that will occur as specified in this chapter, school districts will conduct 
ongoing student assessment and provide information regarding pupil performance based on 
those assessments on a regular basis to parents or guardians and schools.”). 

^^SFUSD’s 3rd Supp. Resp. No. 40, 44; BOHUSD’s Further Supp. Resp. No. 16-17. 
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in the legislative chamber where a bill is being debated. Test anxiety in the 
classroom is not something unique. It is a part, though hopefully not too 
large a part, of life itself. 

For native English speakers, lack of proficiency in English may be relat- 
ed to excessive absences from school, frequent changes in schools, family 
problems, poverty, or other educational issues. There are no provisions in 
the STAR statute or regulations for exempting low-scoring native English 
speakers whose poor English skills create test anxiety. Indeed, the exclusion 
of students from testing because of potential anxiety caused by poor litera- 
cy skills does not serve the students’ long-term interests in effective assess- 
ment of their achievement. Furthermore, exclusion of such students from 
testing would prevent the state from gathering complete data in order to 
hold school districts publicly accountable for all students* progress over time. 

Appropriate Test Preparation 

In the short term, some ELLs<30 not accustomed to testing in English may 
find the SAT9 difficult. However, there is much SFUSD et al. can do to 
ease their transition. In my professional experience, students are sensitive to 
the negative feelings of adults and parents and can develop anxiety about a 
test after overhearing negative comments about the test -from adults. 
SFUSD et al. can counter such pressures by reassuring students and parents 
that test results wiU not be used in any negative manner by the school and 
by emphasizing the ways ELLs<30 v/ill benefit from taking the test each 
year. ELLs<30 can also be told that the test will be challenging, that they 
are not expected to know all the answers, that they are to try their best, that 
they can skip questions they do not understand, and that they should 
answer any items they can while studying the test to find out what it is like. 

Indeed, the test directions reinforce these ideas. A notation labeled “Dear 
Student” on the first page of the test booklet states: “You are not expected 
to know everything on the test, but please try hard to answer the questions.” 
In the oral directions to the student, the examiner says: “If you’re not sure 
about the answer to a question, do the best you can, but don’t spend too 
much time on any one question.” 

In addition, the statutory scheme and SEE regulations make special pro- 
visions for ELLs<12, the subgroup of ELLs<30 for which the SAT9 may 
be most challenging. The Accountability Act provides that the scores of 
ELLs<12 are not counted in a school’s API, and the SEE has adopted a 
policy that allows administration of the SAT9 to ELLs<12 with modifica- 
tions including directions translated into the student’s primary language 
and use of a bilingual dictionary during test administration. 



*Tbel, R.L. 8c Frisbie, D.A. Essentials of educational measurement (4th ed.). Englewood 
Cliffs, NJ: Prentice-Hall (1986), p. 205. 
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In sum, by adopting and communicating a positive demeanor, clearly 
indicating that ELLs<30 are not expected to know all of the tested mate- 
rial, and appropriately implementing the SBE policy on modifications 
for ELLs< 12, SFUSD et al/s educators can create a supportive environ- 
ment in which students and their parents can learn and benefit from the 
testing experience. With careful attention to strategies for easing the 
short-term transition to testing in English for current ELLs<30, SFUSD 
et al. can provide them with a long-term educational benefit. 

Parent Exemptions 

The parent initiated exemption provided for in Tide 5 of the California 
Code of Regulations § 852(a) is similar to that of other states which allow 
individual parents to excuse their children from testing for personal reasons 
(e.g., because the testing conflicts with their religious beliefs). The prohibi- 
tions on solicitation or encouragement of student exemptions do not pre- 
vent SFUSD et al. from providing accurate and appropriate post-adminis- 
tration SAT9 score interpretations to ELLs and their parents. 

The language prohibiting district solicitation or encouragement of 
parental exemption requests is designed to prevent districts from violating 
the spirit of the STAR Program prior to SAT9 testing. Based on my expe- 
rience with statewide testing, I have found that in other states, when dis- 
trict staff encouraged parents to exempt children likely to score poorly on a 
statewide standardized test, their primary concern has been the image of 
the district. In the words of one of SFUSD et al.’s experts, the SFUSD dis- 
trict is seeking to avoid “[t]he inclusion of those scores [that] will do noth- 
ing more than depress school and district averages and wrongly portray 
public education as failing its children.”'*^ 

Summary 

In sum, any potential harms of administering the SAT9 to ELLs <30 are 
not particular to this subgroup and can he avoided or mitigated hy the 
school districts themselves. At best, the evidence for the alleged harms is 
anecdotal and unsubstantiated by objective evidence for ELLs as a group. 
In contrast, as indicated above, there are short and long term benefits of 
testing ELLs<30 with the SAT9 and including them in the state 
accountability system. 

One example of documented positive consequential validity for 
ELLs<30 and ELLs >30 are the gains in state and individual district SAT9 
test scores for ELLs over the three years the STAR Program has been oper- 
ational. These results are presented for SFUSD et al. and three comparison 
districts in a later section. A recent Newsweek article summarized some of 



*^De Avila declaration, p. 7. 
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the consequential validity evidence for testing ELLs as follows: 

Next, examine California’s Proposition 227. Passed June 1998 by a 61 to 39 
percent margin, it banned bilingual education in the state’s schools. 
Educators widely opposed it; so did President Clinton. Prophecies of doom 
were widespread. Clinton said it would condemn immigrant children to 
“intellectual purgatory.” The head of the San Francisco School Board said 
that “this would set our students back 30 years.” 

What happened? Test scores of children from Spanish-speaking families 
didn’t drop. They rose. In second grade, average reading scores of students 
with limited English ability have jumped in the past two years from the 19th 
percentile nationally to the 28th percentile. In math, the same students went 
from the 27th to the 41st percentile, according to The New York Times. 

“I thought it would hurt kids,” Ken Noonan, superintendent of schools in 
Oceanside, a city north of San Diego, told the Times. Thirty years ago he 
helped found the Cahfornia Association of Bilingual Educators. “The exact 
reverse occurred, totally unexpected by me,” he said. “The kids began to 
learn — not pick up, but learn — formal English, oral and written, far more 
quickly than I ever thought they would.”^* 

Reliability 

Test reliability refers to consistency of measurement. A test measures con- 
sistently when students tested on different occasions or with a different test 
form achieve similar scores. Because an individual student may vary in per- 
formance from one day to the next due to differences in motivation, health, 
attitude, environmental conditions, memory lapses, or idiosyncratic reac- 
tions to individual items, test scores contain some errors of measurement 
and scores on two forms of a test will not be identical. However, most stu- 
dents who perform well on the first test vwll also perform well on the sec- 
ond test, and most students who perform poorly the first time will also do 
so the second time. Reliability is measured on a scale of 0 to 1.0, with 0 rep- 
resenting no reliability and 1.0 representing perfect reliability. 

Reliability and validity measure different properties of a test. To provide 
useful information, an achievement test must measure accurately (validly) 
and consistently (reliably). 

Typ es of Reliability Evidence 

There are two major procedures for calculating test reliability: repeated 
testing and measures based on a single test administration. Calculating reli- 
ability based on repeated testing is most important when students take dif- 
ferent forms of an achievement test. For these students’ scores to be com- 
parable, the two forms must measure consistently so that a student will be 
indifferent about which form is administered. This type of reliability is 

■**Samuelson, R.J. The lesson of tough love, Newsweek^ September 2, 2000, p. 27. 
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referred to as alternate forms reliability and is typically estimated by the test 
publisher using a selected sample of students administered both forms of 
the test. 

When large groups of students for whom comparisons are to be made all 
take the same form of an achievement test, such as the SAT9 administered 
in California, a single administration reliability estimate is most practical. 
A single administration reliability estimate can be calculated directly from 
the students’ test scores. For multiple-choice tests with a single correct 
answer for each item, a KR 20 reliability is typically calculated. The KR 20 
reliability was named after two researchers, Kuder and Richardson, whose 
formula for this reliability measure was identified by the number “20” in 
their published paper. 

Kuder and Richardson also developed an approximation to the KR 20 sin- 
gle administration reliability estimate, referred to as KR 21 , which assumes 
that all test items are of equal difficulty. KR 21 requires less computation and 
can be calculated from the mean and standard deviation of the group. 
When the assumption of equal item difficulties is not met, as is typical for 
a nationally-normed achievement test for which the items span a range of 
difficulty, KR 21 will provide a slighdy lower estimate of reliability than 
KR 20 . In comparison to KR 20 and KR 21 , an alternate forms reliability esti- 
mate is usually somewhat lower because it involves two separate test admin- 
istrations for which conditions might be somewhat different. 

Reliability of the SAT9 for All Students 

For purposes of illustration, data for the SAT9 Reading and Math tests for 
Grades 2, 4, 7, and 10 have been selected for inclusion in this report. These 
choices were made to represent the broad array of available data. In partic- 
ular, the SAT9 Reading and Math tests were chosen because reading test 
performance is likely to be affected by English proficiency to a greater 
degree than math test performance. In selecting grades, two elementary, 
one middle school, and one high school grade were chosen to illustrate per- 
formance across the span of grades and to provide more information at the 
elementary level where about 63 percent of the ELLs are located.'^ Grade 
2 is the initial grade for administration of the SAT9, and students respond 
in the test booklet. Grade 4 is in the upper elementary range, and students 
record their answers on separate answer sheets. Grades 7 and 10 are in the 
center of the range for middle school and high school, respectively. 
Together, the SAT9 data for these subjects and grades provide a snapshot 
of the characteristics of the SAT9 test and its implementation in California. 

^CDE, A Packet of Easily Accessible Information on English Learner Students, prepared for the 
English Learner Advisory Committee, January 27,2000, Table 2. Statistics are for 1999 and 
indicate that 45.4 percent of ELLs were in grades K-3 and 62.8 percent of ELLs were in 
grades K-5. 
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KR 2 o> KE -217 alternate forms reliability estimates for the SAT9 are 
obtained from the norming sample. Reliabilities for the Reading and Math 
tests for selected grades are reported. The alternate forms reliabilities range 
from .81-.93, the KR 21 S from .S7-.96, and the KR 20 S from .S8-.96. As 
expected, the alternate forms reliabilities are somewhat lower than the 
KR 20 S and KR 2 iS- These data also indicate that the KR 21 S provide a very 
close approximation to the KR 20 S. 

Reliability of the SAT9 for ELLs<30 

Based on disaggregated SAT9 test score data provided by the districts,'*^ 
KR 21 reliability estimates were calculated for ELLs<30 and ELLs>30 for 
SFUSD, OUSD, and HUSD.'** The same subjects, Reading and Math, and 
the same grades, 2, 4, 7, and 10, were selected for illustration. The results 
are presented in Table 2. 

The SAT9 KR 21 reliability estimates reported for ELLs<30 and 
ELLs>30 are nearly as high as those reported for all students and in a few 
cases are higher. 

In 75 percent of the comparisons for both SFUSD and OUSD, the reli- 
ability for ELLs<30 was slighdy higher than that for ELLs>30. Except for 
OUSD Grade 10, aU of the KR 21 reliability estimates are at least .85 for 
both 1999 and 2000. The lower estimates for OUSD in Grade 10 are based 
on small numbers of students with a restricted range of test scores. OUSD 
tested all ELLs in 2000 but exempted some from Category I (their least 
proficient ELLs) in 1999. However, the reliability estimates for the two 
years are quite similar. HUSD KR 21 S for ELLs<30 are also high and con- 
sistent with those for the other districts. 

Completion Rates 

Completion rates for the SAT9 are high. Therefore, it is appropriate to 
report KR 20 and KR 21 reliability estimates for the SAT9 because the vast 
majority of students are able to finish the test within the prescribed time 
limits. 



^^BUSD’s initial production contained errors and their data analyst was unable to verify the 
accuracy of the revised data that was produced because the designation of ELLs<30 was 
based on a list of unknown origin from another department (Deposition of Marianne 
Spelenda, Vol. II, August 8, 2000, p. 165-67). Due to the questionable accuracy of their data, 
BUSD was not included in the analyses presented in this report. 

**The data provided by the districts did not include the individual item responses necessary to 
calculate a KRjo reliability estimate. However, as the SAT9 data for all students demon- 
strates, KR 21 S are slightly more conservative reliability estimates than KR 2 oS* 
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Standard Error of Measurement 

In addition to considering the reliability of the total test, consistency of 
measurement can also be evaluated based on the standard error of measure- 
ment, The standard error of measurement (SEM) provides an estimate of 
the expected amount of chance fluctuation in individual student test scores. 
For example, for students who score 55 on a test with a SEM of 2, one 
would expect the true achievement of about two thirds of them to fall with- 
in one standard error (between 53 and 57). SEMs for the SAT9 for 
Reading and Math at selected grade levels range from 3 to 4V2 score points, 
relatively small values for tests consisting of 48 to 118 items. 

Evaluating Reliability Evidence 

KR 20 and KR 21 reliability estimates for nationally-normed achievement 
tests are typically in the upper .80s and low .90s. A rule of thumb I have 
recommended for evaluating reliability evidence is that the reliability esti- 
mates be at least .80 when the test data are used in accountability systems 
to evaluate schools and at least .85 when the test data are used to make 
high-stakes decisions for individual students (e.g., promotion or awarding 
a high school diploma). 

In the STAR Program, the test data are used to make high- stakes deci- 
sions about schools but not students, so the appropriate reliability thresh- 
old is the .80 for school level decisions. However, the KR 20 and KR 21 reli- 
ability estimates reported for the SAT9 are all above .85, the more stringent 
criterion for high-stakes student decisions. Except for OUSD Grade 10, all 
of the calculated KR 21 district reliabilities for ELLs<30 are also above the 
.85 threshold. Given the high reliability of individual student scores, the 
aggregation of those scores to form school averages provides a very reliable 
indicator of overall school performance and the performance of particular 
subgroups comprised of adequate numbers of students. These data indi- 
cate that the SAT9 is sufficiently reliable for the purposes for which it is 
being used in the STAR Program. These data also demonstrate that the 
SAT9 provides reliable scores for ELLs<30. 

Variability in English Proficiency 

In the chapter on Reliability, the 1985 Test Standards states: “[T]he impor- 
tance of a particular source of error depends on the specific use of a test” (p. 
19). If the test is intended to distinguish between students who can demon- 
strate specific content skills in English and those who cannot, variability 
among students in English proficiency is part of the achievement being 
measured and is not error. For a standardized test to be reliable, there must 
be some variability in achievement within the group of students for which 
the reliability estimate is obtained. The district data demonstrate that 



READ PERSPECTIVES 

130 





national percentile rank (NPR) scores for ELLs<30 range from the low sin- 
gle digits to the high 90s in most cases and that this range is similar to that 
for ELLs>30. 

These data also indicate that the SAT9 can provide reliable scores for 
ELLs even when significant numbers receive similarly low scores. Thus, 
although SAT9 scores may not provide enough information to determine 
which of two very low-achieving students has learned more, SAT9 scores 
can indicate that both students^ current achievement is well below state 
expectations and the national average. Average SAT9 scores can also reli- 
ably track gains of lower scoring groups over time. 

Reliability Related to Test Purpose 

The STAR Program statute does not contemplate that the SAT9 be used 
to identify ELLs or to prescribe any particular instructional program for an 
individual student. The state sets no cutoff scores for individual students 
and makes no interpretations about why a student performs poorly or what 
should be done about it. The SAT9 test is not designed to predict ELL sta- 
tus; its purpose is only to determine whether a student has achieved the 
tested skills at a particular point in time. If a school district chooses to use 
the SAT9, or any other standardized test, to make inappropriate decisions 
about ELLs, that action does not invalidate the state’s appropriate use of 
the SAT9 to monitor school improvement in educating all students, includ- 
ing ELLs, nor does it affect the reliability of ELLs’ test scores. 

Reliability of Scores for Low- Scoring Students 

Reliability is related to errors of measurement that occur when a student 
guesses correctly but really does not know the tested content or when a stu- 
dent misunderstands a question or is careless and misses an item measuring 
content the student does know. Because most tests contain more average 
difficulty items than very easy or very hard items, errors of measurement are 
slighdy smaller for average students than for high- or low- scoring students. 
Nonetheless, a highly reliable standardized test such as the SAT9 can deter- 
mine the relative proficiency of different groups of students with a high 
degree of accuracy. 

Chance Level Scores 

SFUSD et al.’s witnesses have argued that the SAT9 as administered by the 
state is unreliable because ELLs score at the chance level.'” Such assertions 
are unwarranted for four major reasons: (1) low scores can indicate the 
absence of skills; (2) students do not respond randomly to all achievement 
test items; (3) a fair accountability system requires the inclusion of all stu- 

'•’Declaration of Edward De Avila, April 9, 1998, H 5. 
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dents; and (4) SFUSD et al. data indicate that the vast majority of 
ELLs<30 score above the chance level on the SAT9. 

Low Scores Indicate Absence of Skills 

If the purpose of the test is to measure acquisition of grade level content in 
English and a student can score no better than chance, the student’s low test 
score accurately reflects an absence of the tested skills. A test can validly 
indicate what a student cannot yet do as weU as measuring what a student 
has achieved. 

Scoring at the chance level means that the student obtained the same 
number of correct answers as would be predicted for a student marking 
answers randomly. A student who randomly chooses an answer to a multi- 
ple-choice question with four answer choices will guess correctly about one 
out of four times. On a test consisting of 100 four-choice multiple-choice 
items, a student choosing answers randomly would be expected to correct- 
ly answer about 25 percent of the items by chance. Thus, a student who 
receives a score of 25 on this test has performed similarly to a student who 
marks the answers randomly wathout looking at the test questions. 

When a student can perform no better than chance, it is a valid indica- 
tion that the student’s academic achievement is below that measured by the 
test. This does not indicate a problem with the test but rather indicates that 
the student needs more instruction on the skills measured by the test. For 
ELLs<30, that instructional assistance may include English language 
instruction in addition to content area instruction. 

Students Do Not Respond Randomly to All Achievement Test Items 
The chance model referred to by SFUSD et al.’s witnesses is not an appro- 
priate model for achievement testing because students do not choose 
answers for all the items at random. On weU-constructed achievement tests 
such as the SAT9, students with partial knowledge are able to eliminate one 
or more answer choices or are drawn to incorrect answers reflecting com- 
mon misunderstandings. In addition, students taking tests with no conse- 
quences for them tend to leave items they do not know blank. As a result, 
low-achieving students may receive lower scores than predicted by a chance 
model. These low scores indicate that the student is not yet able to do what 
the test measures. Determining the reasons for the deficiency and how it 
will be remedied are matters for each district to decide. 

Another reason why the chance model is not an appropriate model for 
ELL responses to SAT9 items is that English proficiency is not always 
required to answer SAT9 items correctly. In particular, some math compu- 
tation items require little or no English proficiency. For others, pictures 
provide significant help in understanding what to do. Several examples of 
the types of items for which elementary students with limited English skills 
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who had learned the tested content could be expected to answer correctly, 
even without fully understanding the English words, are contained in the 
SAT9 Guides for Classroom Planning}^ 

A Fair Accountability System Requires Inclusion of All Students 
The state imposes no consequences for individual students, including those 
who score at the chance level. Chance level scorers may be ELLs<30, 
ELLs>30, or native English speakers who have not yet achieved the tested 
skills due to language difficulties or lack of content knowledge. It is impor- 
tant to include such students in the state’s school accountability measure to 
get an accurate estimate of the achievement status of the school so that 
resources can be targeted to those schools that most need them. 

If ELLs<30 who might achieve chance scores due to their language dif- 
ficulties were excluded, it could artificially inflate the school averages and 
make the school appear to be performing acceptably when in fact there was 
great need for instructional resources to assist those ELLs<30 who had not 
yet achieved the tested skills. In addition, if a school is to earn a reward for 
outstanding improvement, that improvement should , reflect improvement 
for the entire student population of that school, not just selected students. 
If the school is doing its job, an ELL <30 ^o scores at the chance level 
this year should score above chance next year. It is appropriate for the 
state to set growth targets based on all students and to provide assistance to 
those schools with the largest populations of underachieving students. 

Most SFUSD et al ELLs Score above Chance 

Chance level scores and their corresponding national percentile ranks for 
SAT9 Reading and Math tests at selected grades are instructive. Except for 
the Math test in Grades 7 and 10, the NPRs for the chance level scores are 
5 or less. This means that a student scoring at the chance level has received 
a higher score than only 5 percent or less of the students in the national 
norm group. 

SFUSD et al. create the impression that most ELLs<30 score at the 
chance level.^’ However, this belief is not supported by the data. The per- 
cents of SFUSD, OUSD, and HUSD ELLs<30 and ELLs>30 scoring 
above the chance level on the 2000 SAT9 Reading and Math tests in 
Grades 2, 4, 7, and 10 demonstrate that the vast majority of ELLs<30 are 
scoring above the chance level. In many cases, the percent of ELLS <30 
scoring above the chance level is similar to or exceeds that of ELLs >30. 

For example, SFUSD ELLs<30 had equal or better results than 
ELLs>30 half the time for Reading and all the time for Math. For OUSD, 

“See Primary 2, p. 10, 13, 16; Intermediate 1/2/3, p. 14, 17; Advanced 1/2, p. 14, 15. 
“Declaration of Edward De Avila, April 9, 1998, H 5. 
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the percent of ELLs<30 scoring above chance vs^as greater than that for 
ELLs>30 in three of the eight comparisons and within 7 percentage points 
in all but one of the remaining comparisons. HUSD ELLs<30 had a slight- 
ly lower percent of students scoring above chance than ELLs>30 and, except 
for Grade 10, were in the high 80s and low 90s. The magnitude of the dif- 
ferences between ELLs<30 and ELLs>30 ranged from 0-8 percentage 
points for SFUSD, 1-10 percentage points for OUSD, and 1-7 percentage 
points for HUSD. These data indicate that the SAT9 is at an appropriate 
level of difficulty for measuring ELLs<30 and that the vast majority of 
these students have learned some of the skills measured by the test. 

Differential Performance 

The term “discrimination,” also known as differential performance, is used 
in multiple psychometric contexts. It can refer to item discrimination, test 
“bias,” or impact data. 

Item Discrimination 

When an achievement test is developed, one of the item statistics used to 
judge item quality measures the degree to which items “discriminate” 
between those who have learned the tested content and those who have not. 
That is, the intent is that students answer items correctly when they have 
learned the tested content but incorrectly if they have not. Use of such item 
statistics in test development assists publishers in identifying and eliminat- 
ing items that are miskeyed, ambiguous, have more than one correct answer, 
or have other flaws that interfere with accurate measurement of a student s 
knowledge and skills. 

Test “Bias” 

Test “bias” is another testing concept related to the term discrimination. 
Measurement experts consider items on a test to be biased if construct irrel- 
evant variance causes differential performance for students in a particular 
racial or ethnic group. SFUSD et al. s experts argue that the SAT9 is biased 
against ELLs<30 because it includes the construct irrelevant variance of 
lack of English proficiency. As explained in the section on validity above, 
English proficiency is part of the skill being measured so differences in 
Eng^sh proficiency among students or groups constitute construct rele- 
vant 

Any student, including a white native English speaker from a low socio- 
economic background, who lacks proficiency in English wiU score poorly. 
ELLs<30 are not being singled out because of their race or ethnicity or 
national origin. Rather, when they receive low test scores it is because they 
cannot yet do what the test measures. Thus, the group of students for whom 
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there will be differential performance on the SAT9 will include all students 
from both minority and majority groups who cannot yet do what the test 
measures. 

Impact Data 

Impact data compare the test performance of majority and minority stu- 
dents. For example, a high school graduation test with a white passing rate 
of 90 percent and an African-American passing rate of 70 percent has a dif- 
ferential impact of 20 percent. 

SFUSD et al. argue that the SAT9 test discriminates against ELLs 
because their average test performance is lower than that for nonELLs. For 
example, a declaration of one of SFUSD et al. s experts reports SAT9 
means by grade for ELLs and nonELLs in Reading and Math for 1998 and 
1999 and uses a statistical test to conclude that the means for these two 
groups are significantly different.” This argument is psychometrically in- 
correct and misconstrues the purpose of the test. 

For example, if students in Group A have learned more of the tested con- 
tent than students in Group B, a weU-constructed achievement test should 
recognize the differential achievement in these two groups by producing 
mean test scores that are higher in Group A than Group B. Thus, if ELLs 
are like the students in Group B because they have not yet fully achieved 
academic proficiency in English, lower average achievement scores for 
ELLs are appropriate and reasonable. 

Once these students’ English skills show enough improvement, they will 
be redesignated fluent English proficient (FEP) and will become part of the 
nonELL group. Thus, the makeup of the ELL group will be constantly 
changing as new students arrive and existing members of the group are 
reclassified. Because the ELL group is a temporary classification for stu- 
dents with the least proficiency in English, one would expect this group to 
continue to score lower on average than students who do not share this 
characteristic. On the other hand, one should be concerned when the test 
scores for the same students classified as ELLs for several years indicate lit- 
tle or no gains in achievement. 

Because an ELL designation indicates a temporary lack of some of the 
skills necessary to perform well on the SAT9, lower performance may occur 
until the student has learned the tested English language and content skills. 
If differences in ELL and nonELL average test scores were judged to con- 
stitute discrimination against ELLs, then by the same logic, the test would 
also discriminate against Title I students (students achieving significantly 
below grade level who receive special help in federally funded programs) 

^Declaration of John Poggio,June 30, 2000, Tables p. 1-2. Results of repeated independent t- 
tests for the difference in 1999 SAT9 ELL and nonELL means across grades yielded t val- 
ues of 211.6-868.6 for Reading and 114.5-589.7 for Math (Tables IB &2B). 
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whose test performance is also lower than students not classified as Tide I. 
Both ELLs and Tide I students have been identified for membership in 
their respective groups based in part on their academic functioning. Until 
that academic functioning changes, it is appropriate that they may achieve 
lower scores on a related academic test. 

One should not expect equal test scores for groups, such as ELL and 
nonELL, which are defined by some of the same academic skills measured 
by the test. However, because ELLs begin with an academic disadvantage, 
ELLs must experience an accelerated rate of growth to catch up with those 
who begin ahead of them. 

Other Low-Scoring Subpopulations 

California state data demonstrate that ELLs are not the only low-perform- 
ing group. SAT9 2000 NPRs for the average student in groups designated 
nonELL, ELL, Economically Disadvantaged, and Special Education are 
reported for Reading and Math in Grades 2, 4, 7, and 10 and indicate sim- 
ilar test performance for ELLs, Economically Disadvantaged, and Special 
Education students statewide. 

Incorrect Statistical Analysis 

In addition to being inappropriate psychometrically, the impact analysis of 
SFUSD et al.’s expert is incorrect because it uses inappropriate statistical 
techniques. Computation of a t-statistic is only appropriate when applied 
to samples of students. The data SFUSD et al. use to make their compar- 
isons are not samples of students but rather the entire population of stu- 
dents tested in California for two consecutive years. In 1999, the number of 
ELL students tested in California ranged from 127,406 in Grade 3 to 
38,202 in Grade 11 With 83 percent of Grade 3 ELLs tested and 74 per- 
cent of Grade 11 ELLs tested.^^ 

When calculations are based on a substantial proportion of the popula- 
tion, the observed differences are reasonable estimates of the actual popu- 
lation differences. No statistical calculations are required to state the 
obvious: aggregated scores for ELLs are lower than aggregated scores for 
nonELLs.^^The important issue is deciding how to interpret this informa- 
tion. 

Unfortunately, due to the large numbers of students in each group, use of 
statistical tests in such situations produces meaningless results that are not 
helpful for describing the practical significance of the observed differ- 
ences.^^ In fact, the use of statistical tests for evaluating impact in large pop- 

”California STAR state summary report, www.cde.ca.gov, 9/13/00. 

^‘Because the ELL students excluded from testing by some districts were probably their least 
able students (in OUSD that was the case in 1999), inclusion of these students’ scores would 
not have changed the result. 
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Illations will indicate significant impact unless the observed difference is 
less than 1 percent, a trivial difference in practical terms. As already indi- 
cated, the lower average group scores of ELLs relative to nonELLs are an 
accurate reflection of the differential skill levels in the two groups at a given 
point in time. 

Improvement of Low-Performing Groups 

In my experience with statewide testing, it is common for new statewide 
testing programs to draw criticism from advocates for “at risk” groups like- 
ly to score poorly on initial test administrations. However, over time, such 
programs typically have recorded substantial achievement gains for at-risk 
populations that would not have occurred absent the public scrutiny and 
accountability associated with testing. 

For example, initial passing rates for minority students on a high school 
graduation test in Texas increased substantially over a five-year period. In 
1994, 29 percent of African-Americans and 35 percent of Hispanics passed 
all tests taken for the first time in tenth grade. By 1998, these percentages 
had risen to 55 percent and 59 percent, respectively^^ 

Similar trends for ELLs are apparent in the three years of data currently 
available for the SAT9. Statewide data summarize SAT9 NPR gains for 
ELLs from 1998 to 2000 in Reading and Math for Grades 2, 4, 7, and 10. 
These data indicate gains in all grades and subjects with the largest gains 
for math in the elementary grades. For example, NPR scores for the aver- 
age ELL student in Grade 2 Reading rose from 19 in 1998 to 28 in 2000; 
for Math the values were 27 and 41, respectively. For Grade 4, the corre- 
sponding values were 15 to 20 for Reading and 21 to 30 for Math. 

Comparisons for SFUSD et al. districts and three other districts^^ which 
have been particularly successful with ELL students are also instructive. 
The demographic data demonstrate that the three comparison districts are 
similar in size, percent of ELLs, and current performance to the SFUSD et 
al. districts. Where the three comparison districts differ most is in the size 
of the gains they have achieved with their ELL students. While the 
SFUSD et al. districts gains are smaller than statewide gains, the three 
comparison districts have SAT9 gains from 1998 to 2000 which are larger 
than the statewide gains. 

”In addition, when the number of students in each group is large, a z-test is more appropriate 
than a t-test. A t-test is designed for small sample comparisons of 30 students or less per 
group. Moreover, when multiple comparisons are being made, a MAN OVA analysis is more 
appropriate than a series of t-tests to constrain experimental error to an appropriately low 
level. 

*H!'EA Statewide Results, www.tea.state.tx.us. 

”Alameda City Unified School District (AUSD), Inglewood Unified School District (lUSD), 
and Oceanside Unified School District (OcUSD). 
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Another way to evaluate the improvement of ELLs in districts is to 
examine the percent of students scoring above the 25th NPR, 50th NPR, 
and 75th NPR. These data for SAT9 2000 indicate, for example, that in 
both Reading and Math, AUSD has a greater percentage of all ELLs above 
the national average in 2000 than either SFUSD or OUSD have for only 
their ELLs>30.This trend also holds for comparisons at the 25th and 75th 
NPRs. 

In sum, both the percents of students scoring above specified NPRs 
and the gains of ELLs from 1998 to 2000 indicate substantial progress for 
ELLs at all score levels in the three comparison districts. The ELL pro- 
grams in AUSD, lUSD, OcUSD, and other similarly effective districts can 
be studied further to determine whether their methods can be exported to 
other districts. 

In addition, ELLs enrolled for less than 12 months (ELLs <1 2) on aver- 
age score substantially above the chance level. The results for ELLs<12 are 
reported by subject and grade for the state, OUSD, and AUSD. For com- 
parison, the NPR corresponding to a chance level score for each subject and 
grade is also available. These data demonstrate that even in OUSD, where 
ELLs <12 score below the state average, their average scores are still above 
the chance level, especially in the elementary grades for reading and all 
grades in Math. The data for AUSD show ELLs<12 performing above the 
state average, including an NPR of the “average” student of 50 in Grade 2 
Reading and 64 in Grade 2 Math. 

Comparing ELLs <30 and ELLs >30 

SFUSD et al.'s claim that ELLs <30 lack sufficient English proficiency for 
the SAT9 would lead one to expect scores for ELLs <30 to be generally 
lower than those for ELLs >30. That is, one would expect the distribution 
of SAT9 scores for ELLs<30 to be shifted lower on the score scale than the 
distribution of SAT9 scores for ELLs >30. However, test score data provid- 
ed by SFUSD et al. for ELLs <30 and ELLs >30 indicate considerable over- 
lap in scores for the two groups. Further, students in both groups obtained 
both very high and very low scores. 

Numbers of Students Enrolled and Tested 

The numbers of ELLs<30 and ELLs>30 enrolled and tested in SFUSD, 
OUSD, and HUSD by subject and grade are available by year, as is the per- 
cent of enrolled students tested. 

For SFUSD in 1994, the percent of ELLs <30 tested ranged from 76 per- 
cent to 90 percent and was higher than the percent of ELLs >30 tested. 
However, by 1998, the percent of ELLs<30 tested had declined sharply to 
a range of 19 percent to 38 percent and was about half that for ELLs>30. 
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By 2000, the percent of ELLs<30 tested was up about 10 percent. 

For OUSD in 1998, the percent of ELLs<30 tested ranged from 18 per- 
cent to 44 percent, similar to that for SFUSD, and was significandy less 
than the percent of ELLs>30 tested. However, by 2000, the percent of 
ELLs<30 tested had risen sharply to a range of 62 percent to 84 percent and 
was much closer to the percent of ELLs>30 tested. 

HUSD demonstrated a similar pattern of increase in the percent of 
ELLs<30 tested from 1998 to 2000. It is also apparent from the data that 
not all ELLs>30 are being tested in any of these districts. In some cases, the 
percent of ELLs>30 tested was as low as 55 percent to 65 percent in 1998 
although OUSD and HUSD were in the mid 70s or higher by 2000. 

The calculations for these findings were based on enrollment data and 
student scores provided by the districts. Because it is possible that the 
enrollment data were collected at a different time than test administration 
data, the percents of students tested may have been affected by students 
changing categories from less than 30 months to more than 30 months. To 
provide an overall picture of percent of ELLs tested, it is necessary to com- 
bine the data for ELLs<30 and ELLs >30 to produce total percents of 
enrolled ELL students tested. For 1998, the percent of ELLs tested ranged 
from a low of 23 percent to 26 percent in Grade 2 for SFUSD and OUSD 
to a high of 64 percent to 73 percent in Grade 7 in these districts. HUSD 
percents of ELLs tested ranged from 38 percent to 61 percent. For 2000, 
the percents were noticeably higher in all districts for each grade and sub- 
ject. These data again demonstrate that even those districts claiming to 
have tested all their ELLs fell far short of 100 percent. 

Test Peformance 

Test data for ELLs>30 and ELLs<30 for SFUSD, OUSD, and HUSD 
were analyzed for Reading and Mathematics for Grades 2, 4, 7, and 10 
across several years for CTBS and SAT9 test scores.^® 

Predictably, the SFUSD data show that the test performance of 
ELLs<30 increased over the years at the elementary level as a smaller per- 
cent of students were tested. Performance of ELLs >30 has also increased 
across the years with 1999 Reading standing out from the other years. 
Scores are also generally higher in Math than in Reading because the Math 
test items tend to be less dependent on language skills. 

The distributions of ELLs <30 and ELLs>30 show a great deal of over- 
lap between the two groups. SFUSD claimed its teachers selected only 
those ELLs with sufficient English language skills to take the SAT9. Thus, 
the SFUSD-tested ELLs<30 are probably a more able group than all 
ELLs<30 in the district. This is demonstrated by the ELLs<30 distribu- 



*®HUSD did not test ELLs<30 prior to 1998 so HUSD comparative data includes only SAT9. 
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tions being shifted slightly to the right of the ELLs>30 distributions and 
by the often higher mean NCE scores for ELLs<30, particularly at the ele- 
mentary grade levels in Reading and all grade levels in Math, Nevertheless, 
there were low scoring ELLs<30 in all years. 

But not all ELLs<30 score in the low range. Not surprisingly, significant 
proportions of SFUSD ELLs<30 score in the average range and some score 
in the high range. 

Moreover, the data for OUSD and HUSD provide similar results. For 
OUSD, the number of ELLs<30 has gone down and back up again since 
1996, Test scores have gone up and then declined in 2000, The pattern for 
ELLs>30 is similar, ELLs<30 and ELLs>30 distributions overlap substan- 
tially and there are average and high scoring students in the ELLs<30 
group. For HUSD, more students are being tested and test performance has 
increased slightly from 1998 to 2000. Again, distributions for ELLs<30 and 
ELLs>30 overlap substantially and there are average and high scoring stu- 
dents in the ELLs<30 group. 

Ethnic and ELL Language Group Differences 

Students in SFUSD et al. s schools are ethnically diverse, and different lan- 
guage minority groups perform at different levels on standardized tests. For 
example, the three largest language minority groups for SFUSD are 
Chinese, Spanish, and Filipino. In 1998, Chinese students comprised 27 
percent of the SFUSD enrollment and 40 percent of ELL students. 
Corresponding statistics were 7 percent and 6 percent for Filipino and 21 
percent and 37 percent for Latino students, respectively.” The 1994-1998 
CTBS data for Reading and Math in Grades 4 and 10 indicate a consistent 
pattern of above average performance for Chinese students, near average 
performance for Filipino students, and below average performance for 
Latino students. 

The performance of SFUSD ELL students by language minority group 
mirrors that for the total group. ELL data for 1994-1998 CTBS Reading 
and Math tests in Grades 2, 4, 7, and 10 indicate that in each case, the 
Chinese ELL students score noticeably higher than the Spanish ELL stu- 
dents. 

In addition, the redesignation rates from ELL to nonELL are higher for 
Chinese ELLs than for Spanish-speaking ELLs. For example, for SFUSD 
in 1999, Chinese ELLs constituted 43 percent of the ELL population but 
57 percent of the redesignations. ^ Corresponding statistics for Spanish lan- 
guage students were 37 percent and 20 percent, respectively. These differ- 
ences across language minority groups may account for the wide range of 

”SFUSD 1998 annual Academic Achievement Report. 

“SFUSD 1999 annual Academic Achievement Report. 
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test scores observed for both ELLs<30 and ELLs>30. 

Similar results emerged from OUSD LEP Achievement Reports. 

Thirty Months Is an 
Arbitrary Exclusion Criterion 

SFUSD et al. claim that ELLs<30 should be exempted from SAT9 testing 
because they have insufficient English language proficiency to score better 
than chance and their scores are therefore invalid and unreliable. However, 
as indicated in the analyses present earlier, the available data do not support 
this assertion. In sum: 

1) Scores for ELLs<30 are as reliable as scores for ELLs>30, comparable in 
reliability to scores for all students in the SAT9 norming group, and 
exceed the stringent .85 rule of thumb for high-stakes decisions for indi- 
vidual students. 

2) The vast majority of ELLs<30 score above the chance level. 

3) Gains for ELL students in three comparison districts at the elementary 
level are substantial, above the state average, and based on testing 95 per- 
cent or more of the students in those districts. In contrast, SFUSD et al. 
are achieving much smaller or negative gains testing 67 percent-93 per- 
cent of their ELLs. 

4) Distributions of SFUSD et al. test scores for ELLs<30 and ELLs>30 
show substantial overlap, indicating that the 30-month criterion does 
not distinguish lower-scoring students. 

5) The range of smallest to largest scores is also similar for both groups and 
comparisons of students by category indicate that there are high and low 
scoring students in both groups. 

6) A study by the Accountability Advisory Committee indicated no line of 
demarcation in time of enrollment that would distinguish ELLs with 
high and low scores on the test.“ 

7) Less than 30 months of enrollment in the district is a poor proxy meas- 
ure of English language proficiency. Based on an English language pro- 
ficiency assessment instrument (LALAR) for ELLs under development 
by SFUSD, the relationship between number of months of enrollment 
and LALAR total score is negligible. The data indicate that correlations 
in grades 2 and 4 are .13 and .23, respectively, also indicating that length 
of enrollment accounts for at most about 5 percent of the variance in lan- 
guage proficiency scores. The remaining 95 percent of the variance in 

'’’Deposition of William Padia, May 2, 2000, p. 74-75. 
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scores is accounted for by other factors. 

8) ELLs enrolled for less than 12 months on average score substantially 
higher than chance statewide and near average for their grade level in 
some districts. 

For these reasons, 30 months is an arbitrary exclusion criterion and is not 
supported by the available data. 



Inappropriateness of Applying 
NAEP Criteria 

The Assessment Act provides that in adopting assessments, California 
should: 

Develop assessments that are comparable to the National Assessment of 
Educational Progress [NAEP] and other national and international assessment 
efforts, so that California s local and state test results are reported in a man- 
ner that corresponds to the national test results." 

When interpreting the directive to develop assessments comparable to 
NAEP, several important factors must be considered. NAEP and SAT9 are 
distinguishable in important respects in that NAEP is not administered 
annually to all students in grades 2 through 11, does not produce individ- 
ual student scores, does not report student scores to parents or schools, does 
not measure longitudinal growth of individual students or schools, and is 
not used to allocate rewards or remedial resources to schools or districts. 
These distinctions reflect the difference in purpose between the two assess- 
ments. Thus, from a psychometric perspective, the reference to NAEP in 
the STAR statute may be logically construed to refer to the production of 
national comparisons and not to mean that the selected tests mirror NAEP 
in every respect. 

Further, according to documentation of the National Assessment 
Governing Board (NAGB) meeting, the NAEP decision to change the 
exclusion criteria from a two- to a three-year enrollment requirement was 
not based on empirical research but on lobbying by those who wanted to 
increase ELL participation in NAEP by offering more “accomodations” and 
primary language testing in Spanish. The change in exclusion criteria was 
part of the compromise that resulted. It did not constitute a finding by 
NAEP that ELLs in general should be excluded from standardized testing 
for a specific time period. The net effect of the compromise was to exclude 
more ELL students from the main NAEP sample while increasing their 
participation in the “accommodations” research sample. 

“CEC § 60602(a)(5), emphasis added. 
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In addition, the number of ELL students involved in the NAEP decision 
was small. NAEP estimated that about 3 percent of eighth graders in reg- 
ular schools were ELL in 1992 and about two- thirds were excluded from 
NAEP testing. This resulted in about 2 percent of the population being 
excluded due to ELL status. In contrast, in California, about 25 percent of 
its students are ELLs. Excluding two-thirds of those students would elim- 
inate 16 percent of the student population from the accountability system. 
Obviously, under the new NAEP exclusion criteria, the proportion of ELLs 
excluded in California would be even higher. 

Alternatives to Statewide Testing 

No test is perfect or completely error free. But before deciding to eliminate 
a particular test use, one must consider the alternatives for decision-mak- 
ing. Without objective test information, decision-makers may be forced to 
rely on data that are less valid, less reliable, more prone to unidentified sub- 
jective biases, and less helpful for the intended purpose. 

For example, if use of the SAT9 were proscribed for evaluating the 
achievement of ELLs <30, district personnel would still have to decide 
whether their ELLs<30 were making adequate progress. They could rely on 
teacher evaluations, but the survey data of ELL classroom tests produced 
by SFUSD et al. indicate clearly that there is no consensus among teachers 
within a district about whether tests should be given in English to 
ELLs<30s, if so what those tests should cover, whether and what modifica- 
tions should be made, or whether these students should be given a second 
chance to take the same test. In grading ELLs<30, these results suggest that 
the criteria used and the weight given to each would vary considerably 
across teachers. 

Research has demonstrated that when assigning grades, teachers consid- 
er such factors as attitude, effort, improvement, and attendance in addition 
to achievement of subject matter content. A recent article in Education Week 
reported a geometry teacher who gave extra credit to students who brought 
canned goods for the food bank. Inclusion of such factors in student grades 
renders grades a poor substitute for tests designed to measure specific sub- 
ject matter skills. While useful for certain purposes, dependence only on 
teacher evaluations of ELLs would produce data that are not comparable at 
the district or school level and may not even be comparable within class- 
rooms. Attempts to aggregate such data would provide misleading, incom- 
plete, and inaccurate measures of the progress of ELLs<30 within a district. 

Alternatively, the district could avoid collecting any evidence at the dis- 
trict level and rely on schools and teachers to monitor instructional effec- 
tiveness. However, leaving such decisions to the schools is contrary to leg- 
islative policy and creates a strong probability of lack of uniform treatment 
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of ELLs<30 across the district. 



SABE2 Testing 

The Spanish Assessment of Basic Education, Second Edition (SABE2) 
measures academic skills in Spanish. The 1994 SABE2 Technical Report 
states that the purpose of the test is to evaluate the success of bilingual pro- 
grams and to evaluate student progress in bilingual programs,^ The pub- 
lisher states that the content of the SABE2 is based on common educa- 
tional objectives from state and district Spanish-language curricula and that 
the test is designed to be used with students for whom the language of 
instruction is Spanish. 

In areas with minimal language usage, such as math computation, the 
publisher borrowed items and item content from its English language 
achievement test. But for areas heavily influenced by the nature of the lan- 
guage, such as word attack, vocabulary, comprehension, spelling, and 
expression, unique items were written to match appropriate Spanish lan- 
guage objectives and to provide appropriate difficulty and naturalness of 
language. 

Just as a low score on the SAT9 indicates that the student cannot yet 
demonstrate the academic skills it measures in English, a low score on the 
SABE2 indicates that the student cannot yet demonstrate the academic 
skills it measures in Spanish. However, because the two tests measure dif- 
ferent academic skills, one cannot compare the performance of one student 
in English and another in Spanish and determine which student has 
demonstrated greater academic achievement. 

The content specifications for the SABE2 are different from the content 
specifications for the corresponding subject matter tests of the SAT9. 
Because the SABE2 and SAT9 measure different skills, SABE2 scores can- 
not be aggregated with SAT9 scores to produce grade, school, district, 
county, or statewide summary results as CEC § 60643 requires. 

In addition, there are no “truly ^ national norms for the SABE2. The pub- 
lisher obtained a sample of approximately 1,000 Spanish language students 
drawn from nine states, but this did not constitute a random sample of 
Spanish-speaking students in bilingual programs in the U.S.^As a result, 
parents receiving only scores from the SABE2 have limited normative 
information for comparing the progress of their students. 

Linkage to CTBS4 Norms 

The publisher of the SABE2 used a unique research design to link SABE2 
scores to the national norms for the CTBS4. However, the design requires 

1 . 

®*SABE2 technical report, p, 23. 
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a number of strong assumptions that may not be justified. For example, a 
bilingual sample was used to establish the equivalency of item difficulty for 
matched pairs of Spanish and English items. This equivalency is depend- 
ent on the assumption that the bilingual sample group is equally skilled in 
the two languages. This assumption is unlikely to be met in practice because 
students who speak and understand two languages are usually more profi- 
cient in one than the other. Thus, relative differences in language profi- 
ciency may mask differences in difficulty of the content tested. In addition, 
expert raters judging item difficulty of English items tend to differ signifi- 
candy from subsequent empirical data. Judging the equivalency in difficul- 
ty of English and Spanish items would appear to be an even more difficult 
task. 

More importandy, even if the linking were defensible, an interpretation 
of comparable achievement is not. A student who scores at the CTBS 50th 
percentile based on SABE2 performance has responded entirely in Spanish. 
Performance in Spanish provides no information about how the student 
would have performed on different CTBS content in English. 

Conclusion 

For the reasons presented in detail in this report, it is my professional opin- 
ion that the testing of ELLs<30 in California is consistent with applicable 
professional standards, that the 30-month cutoff is an arbitrary value 
unsupported by available data, and that ELLs<30 benefit from inclusion in 
the STAR Program. 
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Settlement Agreement 



T his Settlement Agreement and Release (referred to herein as the 
“Agreement”) is entered into as of the effective date as set forth 
in paragraph 24, by and between: (1) the California Department 
of Education and the State Superintendent of Public Instruction 
(collectively, “CDE”); (2) the California State Board of Education (“SBE”); 
(3) San Francisco Unified School District, its Superintendent, named 
Board members (collectively “SFUSD”); (4) Berkeley Unified School Dis- 
trict and its Superintendent (collectively “BUSD”); (5) Oakland Unified 
School District and its Superintendent (collectively, “OUSD”); (6) 
Hayward Unified School District and its Superintendent (collectively 
“HUSD”); and (7) Real Parties in Interest Carlos Berviz (hereafter 
“Berviz”), hereinafter collectively “the Parties” or “Party.” 

WHEREIAS, On or about April 2, 1998, CDE filed a complaint for 
declaratory relief and petition for peremptory writ of mandate against 
SFUSD in the California Superior Court for the City and County of San 
Francisco, case number 994049 (the “Action”), which was amended to 
include SBE as a plaintiff/petitioner; and, 

WHEREIAS, On or about May 18, 1998, SFUSD filed on behalf of 
itself a cross-complaint against CDE and SBE in the Action, which has 
been amended from time to time; and 
WHEREAS, On or about June 25, 1998, OUSD, BUSD, and HUSD 
intervened and filed a complaint on behalf of themselves against CDE and 
SBE in the Action, which has been amended from time to time; and 
WHEREIAS, On or about May 8, 1998, Carlos Berviz and Jose Aviles 
intervened and filed a complaint against CDE and SBE in the Action on 
behalf of themselves and all students attending SFUSD who are limited 
English proficient and who have attended California public schools for less 
than 30 months; and 

WHEREIAS, the Parties, expressly denying and disclaiming any liabili- 
ty or wrongdoing whatsoever, wish to avoid the extensive time and further 
costs of litigation and, upon full satisfaction of the terms of this Agreement, 
to fully and finally settle and discharge all claims asserted by any Party in 
the Action; 

NOW THEREFORE, The Parties have reached this Agreement as a 
full and complete settlement of all disputes and claims stated in the Action. 
For and in consideration of the valuable covenants and consideration 
described herein, receipt of which is hereby acknowledged, the Parties agree 
as follows: 



READ PERSPECTIVES 

146 





1. SFUSD, OUSD, BUSD, HUSD AND Berviz agree that all English 
Language Learner (“ELL”) students enrolled in California public schools 
in grades 2-11 shall take the achievement tests under the Standardized 
Testing and Reporting (“STAR”) Program in English, specifically includ- 
ing the standards based test and the norm based test and their successor 
tests, subject only to applicable lEPs or statutory exemptions. 

2. SBE will amend Title 5, Code of California Regulations, Section 
852(a) to state in its entirety: “A parent or guardian may submit to the 
school a written request to excuse his or her child from any or aU parts of 
any test provided pursuant to Education Code section 60640. A school dis- 
trict and its employees may discuss the STAR Program with parents and 
may inform parents of the availability of exemptions under Education Code 
60615. However, the school district and its employees shall not solicit or 
encourage any written exemption request on behalf of any child or group of 
children.” 

3. Where the basis for denying rewards, implementing interventions or 
taking any other action affecting a school or district under the Public 
School Accountability Act is shown by a district to be solely attributable to 
the scores of ELL students, the district, on its ovm behalf or on behalf of a 
school, may submit a request for a general waiver to the State Board of 
Education pursuant to Education Code Section 33050 based on such facts. 
CDE will recommend that the SBE as a matter of policy consider favor- 
ably, among other factors which the SBE or CDE may consider, a showing 
of improvement in student performance on the ELD test, when imple- 
mented. The SBE agrees to use its best efforts to implement and adopt the 
policy set forth in this Agreement as soon as reasonably possible. 

4. SBE and CDE agree that administration of the STAR Program tests 
with the use of standard or non-standard accommodations as set forth in 
Title 5 of the California Code of Regulations, Section 853 will constitute 
“participation” for the purpose of eligibility in the Governors Performance 
Award Program and other awards or grant programs based on the 
Academic Performance Index, subject to applicable law as set forth in 
Education Code sections 400 et seq., 44650 et seq. and 52050 et seq. 

5. SBE and CDE agree to include in the State Superintendent’s annual 
report under Education Code Section 60630 a report of the primary-lan- 
guage academic achievement tests to the extent that districts administer 
such tests. 

6. SBE CDE agree that the State Superintendent’s guidelines for pro- 
fessional development under Education Code Section 60640(c) will include 
guidelines for the use of ELL student STAR Program scores. These guide- 
lines win state that the STAR Program scores of both ELL and non-ELL 
students may reflect the student’s English language ability and his or her 
academic knowledge, and will advise educators that it is best educational 
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practice to rely upon multiple independent information sources (including 
without limitation primary language test scores, report cards, student port- 
folios, and teacher observations) when modifying instruction for the pur- 
pose of improving pupil learning. 

7. The Parties agree that Judge Paul Alvarado of the Superior Court for 
the City and County of San Francisco will determine what amount of costs 
and attorney’s fees, if any, SFUSD, OUSD, BUSD, HUSD and Berviz may 
recover from SBE nd CDE, pursuant to noticed motion, the parties’ papers 
and oral argument. The parties further agree that such recovery, if any, shall 
not exceed $150,000.00 for costs and $150,000.00 for attorney’s fees in 
total for this Action. 

8. SBE and CDE agree that they understand that SFUSD, OUSD, 
BUSD, and HUSD will receive 1999-2000 ELAP funding under Item 
6110-485, subdivision (q) of Chapter 52 of Statutes 2000. 

9. To the extent permitted by law and subject to judicial approval, SBE, 
CDE and Berviz stipulate to certification of a settlement class consisting of 
all ELLs ^vith less than 30 months of public school instruction enrolled in 
California public schools. The settlement class approved by the Court shall 
be bound by the terms of this Agreement, including dismissal with preju- 
dice of any claims set forth in Berviz complaint, subject to notice and an 
opportunity to opt out. CDE and SBE shall bear all costs related to certi- 
fication of and notice to the settlement class approved by the Court. SBE, 
CDE and Berviz further stipulate that any notice required for a settlement 
class shall consist of publication in newspapers of general circulation as 
agreed upon by SBE, CDE and Berviz, including newspapers in languages 
other than English, and shall further consist of written notice sent to school 
districts. Scope of publication issues, if any, shall be submitted to the 
Court’s jurisdiction. The text of any required notice shall be prepared by 
SBE and CDE and forwarded to Berviz for review and comment and shall 
be approved by the Court before publication. SBE and CDE agree to reim- 
burse Beviz for reasonable attorneys’ fees and costs incurred in the certifi- 
cation of the settlement class not to exceed $15,000. 

10. Within five days after the Effective Date of this Agreement as set 
forth in paragraph 24, each Party shall file all necessary papers for dismissal 
with prejudice of the Action and each and every separate complaint, peti- 
tion and cross-complaint therein. Each Party will simultaneously mail a 
file-endorsed copy of the filed dismissal papers to each other Party. 

11. Subject to Paragraph 7 above, each Party hereby releases, acquits and 
forever discharges each other Party and its board members from any and all 
claims, demands, actions, causes of action, suits, obligations, controversies, 
expenses, costs, and fees of any type or nature whatsoever, in equity or at 
law, by statute or common law, alleged in this Action. 

12. The Parties agree that this Agreement is subject to the provisions of 
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Code of Civil Procedure Section 664.6. The Parties stipulate that Judge A. 
James Robertson II of the Superior Court for the City and County of San 
Francisco shall retain jurisdiction over the Parties to enforce this 
Agreement and decide any dispute arising under it. In the event that it is 
necessary for any Party hereto, or its authorized representative, successor or 
assign, to institute suit to compel performance of any of the obligations 
contained herein or to preclude a purported violation of the terms of this 
Agreement, the prevailing Party in such suit shall be entitled to reimburse- 
ment for reasonable costs, expenses, and attorneys’ fees incurred by it in 
such suit, subject to the requirements of Code of Civil Procedure sections 
1021.5 and 1032. 

13. The Parties agree that neither the existence of this Agreement nor 
any of its terms nor compliance with this Agreement shall be taken or con- 
strued to be an admission of any sort on the part of any Party. 

14. Except as otherwise provided herein, each Party fiilly understands 
that if the facts with respect to which this Agreement is executed be found 
hereafter to be different from the facts now believed to be true, it expressly 
accepts and assumes the risk of such possible difference in facts and agrees 
that this Agreement shall be and remain effective notwithstanding such 
difference in facts. 

15. This Agreement may be pleaded as a full and complete defense to, 
and may be used as a basis for an injunction against any action, suit or other 
proceeding of any sort that may be instituted, prosecuted or attempted by 
any Party in breach of this Agreement. 

16. This Agreement shall be binding upon and shall inure to the benefit 
of all Parties and of all class members certified pursuant to Paragraph 9 of 
this Agreement. 

17. This Agreement is in lieu of, supersedes and extinguishes all other 
agreements, negotiations, understandings, and representations which may 
have been made or entered into by and between the Parties and constitutes 
the entire Agreement between the Parties. It is expressly understood and 
agreed that this Agreement may not be altered, amended, modified or oth- 
erwise changed in any respect or particular whatsoever except by a writing 
duly executed by an authorized representative of the Parties. 

18. This Agreement shall be governed by California law. 

19. This Agreement shall be interpreted and construed neutrally in 
accordance with the plain meaning of the language contained herein and 
shall not be presumptively construed against the drafters. 

20. This Agreement may be executed in counterparts, each of which shall 
be deemed to be an original, but such counterparts together shall constitute 
one and the same instrument. 

21. Each Pary acknowledges and declares that it has read this 
Agreement, that it knows and understands the contents of the Agreement, 
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that it fully understands and appreciates the words and terms and effect of 
this Agreement, and that this Agreement has been executed freely, know- 
ingly and voluntarily. 

22. The Parties declare and represent that they have reviewed this 
Agreement in its entirety and in making this Agreement they have relied 
solely upon their own judgment, belief, knowledge, investigation, inde- 
pendent legal advice and research. Each Party acknowledges and declares 
that it has been represented by counsel throughout the litigation of the 
Action and with respect to the negotiation, execution and implementation 
of this Agreement and all matters covered by and relating to the Action and 
this Agreement, and that it has had the opportunity to be, and has been, 
fully advised by said attorneys with respect to its rights and obligations and 
with respect to the consequences of executing this Agreement. 

23. This Agreement is contingent upon approval of the California De- 
partment of Finance, to the extent required by law, 

24. The “effective date” of this Agreement is the date on which it is 
signed by the last Party referred to below or the Court certification of the 
subject settlement class as set forth in Paragaph 9, whichever is later. 

Editor's Note: During the month of November 2000, all parties to the Agreement 
signed the document. 
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All That Glitters 
Is Not Gold 



The Limits of the California Department 
of Education's English Learner 
Achievement Data 

Christine H. Rossell 

C alifornia has a 26-year history of bilingual education that was 
supposed to end with the implementation of Proposition 227 in 
the fall of 1998. The law required that children who were “Eng- 
lish learners” (formerly called limited English proficient or LEP) 
be placed in a sheltered English immersion program in which nearly all 
instruction was in English but at a pace the child could understand. 

Proposition 227 did not completely replace bilingual education because 
parents had the option of requesting that their children be placed in bilin- 
gual education after 30 days in a structured immersion classroom. As doc- 
umented in Rossell (2001), the number of limited English proficient or 
English Learner (EL) students enrolled in bilingual education declined 
from 409,879 in 1997-98 to 169,440 in 1998-99 and then increased slight- 
ly to 169,929 in 1999-00.^ The percentage enrolled in bilingual education 
declined from 29 to 12 percent to 11 percent across aU grades. At the ele- 
mentary level, the percentage enrolled in bilingual education declined from 
39 percent to 15 percent and then increased slightly in 1999-00 to 16 per- 
cent. Similar declines were seen in most of the largest school districts in 
California, although San Francisco and Oakland are notable exceptions. 

Most people, however, want to know not only the extent to which 
Proposition 227 actually eliminated bilingual education, but also the extent 
to which it increased achievement. After all, Proposition 227 was predicat- 
ed on the assumption that English Learners had been greatly harmed by 
bilingual education. Article I, section 300(d) states: 

’These data come from the California Department of Education (CDE) webpage, 
www.cde.ca.gov/demographics. 
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WHEREAS the public schools of California currently do a poor job of edu- 
cating immigrant children, wasting financial resources on costly experimen- 
tal language programs whose failure over the past two decades is demon- 
strated by the current high drop-out rates and low English literacy levels of 
many immigrant children;,.. 

But assessing the effect of Proposition 227, or alternatively of maintain- 
ing a bilingual education program, turns out to be quite difficult, despite the 
fact that the state test, the Stanford Achievement Test (SAT 9), is mandat- 
ed for virtually everyone, including limited English proficient students,^ 
and the test was administered the year before Proposition 227 as well as 
after, thus making pre-post comparisons possible. Moreover, the achieve- 
ment data is publicly available on the website of the California Department 
of Education (CDE).^ Not surprisingly, there has been a rush to draw con- 
clusions from it. The purpose of this paper is to explain the limitations of 
the California achievement data and to critique Norm Gold’s (2000) study, 
“Bilingual Schools Make Exceptional Gains on the State’s Academic 
Performance Index (API),” one of many that ignores these limits. 

No Test Scores for English Learner Programs 

The state achievement data is kept by school and school district and disag- 
gregated by LEP status, gender, economic status (advantaged, disadvan- 
taged), special education status, and grade, but not by program or ethnic 
group. Amazingly, in light of all the publicity given to test scores after 
Proposition 227, it turns out it is not possible to compare the achievement 
of LEP students in a bilingual education program in a school to the 
achievement of LEP students in a structured English immersion program 
in the same school. Nor is it possible to compare the achievement of 
Hispanic LEP students, who typically are the LEP students enrolled in 
bilingual education, to that of other LEP students. This is a huge problem 
that makes it exceedingly difficult, and perhaps impossible, to draw conclu- 
sions from the CDE data about the relative effectiveness of all-English 
instruction or bilingual education. 

Testing Rates 

Equally as problematic is the fact that, although the state requires that aU 
students be tested, this is in fact not occurring for English Learners. 
Approximately 94 percent of all students in California are tested annually 
on the SAT 9, which was first administered in 1997-98, the year before 
Proposition 227. As shownn in Figure 1, however, only 89 percent of ele- 
mentary English Learners and only 87 percent of all English Learners were 
tested in 1999-00, despite the state law requiring it. 

^Special education students with Individual Education Plans are exempted. 

^ http://star. cde. ca.gov. 
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The loophole in the state law is that parents have the right to remove 
their child from testing. Of course, given the lack of knowledge that par- 
ents typically have of such school activities, it is quite likely that the schools 
ask parents to remove their child from testing rather than the other way 
around. 

These data overestimate the testing rates of LEP children because the 
LEP enrollment count comes from an earlier point in time than the test 
scores. Although the tests are taken in May, the LEP enrollment count 
could be taken anywhere from October (when the CBEDS enrollment 
reports are due) through March (when the Language Census reports are 
due). If school districts consistently reported Spring LEP enrollment 
(which they do not), the enrollment would be higher and the testing rate 
lower. 

Two conclusions can be drawn from these data. First, there has been an 
increase in English Learner testing since 1997-98 that could offset any 
gains in achievement resulting from Proposition 227. If more students are 
tested, scores will go down, all other things being equal. Even if each indi- 
vidual student’s scores went up, the aggregate outcome might be zero 
because of the inclusion of more low scorers in the later point in time. 
Second, not all English Learners are tested and this varies considerably by 
school district,^ Comparing schools, or school districts, with different test- 
ing rates is the equivalent of comparing apples and oranges. 

Testing Rates by Program 

There is also a testing rate bias favoring bilingual education programs. 
Teachers can decide when their English Learners are ready to take stan- 
dardized achievement tests and they tend to exempt more bilingual educa- 
tion students from testing than are exempted in alternative programs. In 
general, bilingual education advocates, teachers, and administrators believe 
that it is unfair to administer English language tests to students during the 
period when they are acquiring literacy in their native tongue. 

Although there is a certain logic to this argument, from a program eval- 
uation standpoint it is a disaster. It gives the bilingual education programs 
an unfair advantage over all-English programs since it is the lowest scoring 
students who are deemed not ready to be tested. Thus, a larger number of 

'The state Board of Education has taken notice of the fact that not all students are being test- 
ed, but instead of focusing on the group that is not being tested — English Learners — they 
have adopted regulations (Article 1.7, sections 1031-1038, subchapter 4, Chapter 2, Division 
1, Title 5 of the California Code of Regulations) that require that schools must have 85 per- 
cent of their students tested in spring 2000 and 90 percent tested in spring 2001 or they are 
ineligible for state performance awards. This may have some effect on the English Learner 
testing rate, but it may not have a large effect since it is possible to obtain a 90 percent test- 
ing rate for all students, but still have a much lower testing rate for English Learners if the 
latter group is not a large portion of the district. 
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low achieving students will be excluded in a bilingual education program 
evaluation than will be excluded in an all-English program evaluation. 

Individual Data. Analyses of individual student data, only available from 
the school districts themselves, show this clearly. Ramirez et al.’s (1991) 
national study reported that 89 percent of the structured immersion stu- 
dents were tested in the K-1 analysis, but only 61 percent of the early exit 
bilingual education students were tested. In the grade 1-3 analyses, 42 per- 
cent of the immersion strategy students were tested, but only 29 percent of 
the early exit bilingual education students were tested. 

There is recent evidence from California of even larger disparities. The 
Los Angeles Unified School District, for example, released a study in 1998 
showing English Learners who were in bilingual education for five years 
outscored English Learners in all-English classes on the SAT 9. However, 
as showm on the left side of Figure 2, only 61 percent of the students in the 
bilingual program were thought to know enough English after five years to 
be able to take the test, but 97 percent of the students in the English lan- 
guage program took the test (Los Angeles Unified, 1998). Thus, the so- 
called superiority of bilingual education could easily have been an artifact 
of the exclusion of 36 percent more students with low scores from the bilin- 
gual program evaluation than the all-English program evaluation. 

Bali (2000) found similar results in Pasadena, despite the fact that by 
1997-98, the SAT 9 was mandated for all English Learners. As shown on 
the right side of Figure 2, the testing rate in Pasadena for the English 
Learners in bilingual education was 50 percent. The testing rate for those 
in English as a Second Language, by contrast, was 89 percent, almost twice 
as high as for bilingual education. 

Aggregate Data. The bias in favor of bilingual education is also e\ddent in 
the CDE aggregate data. Figure 3 shows the percentage of elementary 
English Learners taking the SAT 9 in California schools by the extent of 
enrollment in bilingual education in that school, using CDE data. As 
shown, in 1997-98, the percentage of English Learners being tested is 81 
percent in a school vwth no bilingual education enrollment compared to 78 
percent in a school with more than 120 students enrolled in bilingual edu- 
cation. This is further reduced to 76 percent when the bilingual education 
enrollment is above 240 and the percentage enrolled in bilingual education 
is above 40 percent. The testing gap between no bilingual education and 
some bilingual education has increased with Proposition 227, although the 
percentage tested has also increased across all categories. These data suggest 
that bilingual education enrollment in a school depresses the percentage 
tested, although we do not see the large differences e\ddent in the individ- 
ual data.^ 

*The individual student data are the preferred data since they allow one to link test scores to 
individual student characteristics, including program enrollment. 
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Table 1 presents a more extensive analysis of the relationship between 
bilingual education enrollment and school testing rates, controlling for the 
number of Spanish-speaking English Learners, the total school enrollment, 
and the percentage of the school eligible for welfare. This is done for each 
of the three years. The easiest way to interpret the relative strength of these 
variables is to look at the Beta, which is the standardized coefficient. The 
number enrolled in bilingual education has a significant, negative effect on 
the percentage of English Learners who are tested. 

Figure 4 illustrates, for each test year, the reduction in the percentage of 
English Learners tested with each additional percentage enrolled in bilin- 
gual education controlling for the variables in Table 1. The bars are the b 
coefficients for bilingual education enrollment shown in the equations. For 
each additional student enrolled in bilingual education, there is a 3/lOOth, 
or 4/lOOth, of a percent reduction in the percentage of English Learners. 
Thus, in 1999-00, if a school increased its bilingual education enrollment 
from 0 to 33, we would expect the percentage of English Learners who are 
tested to decline by about one percentage point. 



Successful “Bilingual” Schools 

Some people are unaware of the problems with the aggregate data and the 
biases in test scores, while others just ignore them. One example of this is 
Norm GoldV analysis of 63 successful “bilingual” schools (Gold, 2000). He 
concludes there were remarkable gains in the API (Academic Performance 
Index) for the school as a whole, and the Hispanic population in particular, 
that can be attributed to their successful bilingual education programs.^ 
We can assess whether such a conclusion is reasonable by examining 
bilingual education enrollment and testing rates in these schools using the 
same CDE database that Gold used. The school by school data are shown 
in Appendix 1. The first column is the school enrollment. This, and the 
percentage of English Learners enrolled in bilingual education, are the only 
pieces of information in Appendix 1 that are also found in Gold s report. 
The second column is the number of Hispanic students in a school, and the 
third column is the number of Hispanic students enrolled in the tested 
grades. Because the SAT 9 is administered only to grades 2-11, it is neces- 
sary to determine enrollment in the tested grades in order to adjust for non- 
testing. 

The fourth column is the number of English Learners enrolled in the 
tested grades, the fifth column is the number of English Learners tested, 

‘Norm Gold is the retired, former Manager of the Language Proficiency and Academic 
Accountability (LPAA) Unit in the California Department of Education. 

’The report can be obtained from http://www.californiatomorrow.org/files/pdfs/APL 
REPORT_nNAL^12-5.PDR 
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the sixth column is the total bilingual education enrollment, and the sev- 
enth column is the estimated® bilingual education enrollment in the tested 
grades. 

The eighth column shows the percentage of students in the school 
enrolled in bilingual education. This important piece of information is 
missing from Gold’s report, but it is crucial to understanding how achieve- 
ment growth for the school might be influenced by bilingual education 
enrollment. The answer, shovsm in column seven, is that it could not possi- 
bly have much influence on school achievement since only about a third of 
the students are enrolled in bilingual education in these so-called success- 
ful “bilingual” schools. Indeed, if we look dovm the column at individual 
schools, we can see that many schools have less than a quarter of their stu- 
dents enrolled in bilingual education and one school has none enrolled in 
bilingual education. 

Column nine shows the maximum possible percentage of Hispanic stu- 
dents enrolled in bilingual education, assuming that the only students 
enrolled in bilingual education are Hispanic. This crucial piece of informa- 
tion is also missing from Gold’s study. Even if we assume that all the stu- 
dents enrolled in bilingual education are Hispanic, the maximum possible 
percentage of Hispanic students who could be enrolled in bilingual educa- 
tion is less than half in these so-called successful “bilingual” schools. 

Gold does note in his report that the percentage of English Learners 
enrolled in bilingual education is 62 percent. On the one hand, this is irrel- 
evant since he does not examine English Learner achievement growth; on 
the other hand, it does tell us that he was not completely unaware of how 
few students are enrolled in bilingual education in these schools. Sixty-two 
percent is low and it is only simple logic that student rates and Hispanic 
rates of enroUent in bilingual education will be even lower. 

In fact, however, although the percentage of English Learners enrolled in 
bilingual education in his schools is low by his ovm count, it is even lower 
than he thinks it is. The discrepancy between Gold’s calculation of the per- 
centage of English Learners enrolled in bilingual education (column 10) 
and my calculation (column 11) stems from the fact that there are two 
sources for bilingual education enrollment in the CDE database. In column 
11, I use the data from Category 03 in row 02 of the CDE Language 
Census, which is the category that has been used by the state for decades. 
The Census instructions state: 



“Another flaw of the CDE database is that bilingual education enrollment, which comes from 
a different web page than the test scores, is not kept by grade. Since only grades 2-1 1 are test- 
ed, the bilingual education enrollment for those grades has to be estimated in order to calcu- 
late the percentage who might not be tested. The bilingual education enrollment for the test- 
ed grades is calculated by multiplying the total bilingual education enrollment by the propor- 
tion the tested grade LEP enrollment is to the total LEP enrollment. 
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(03) ELD and Academic Subjects Through the Primary Language (Ll): 
These are EL students receiving ELD and, at a minimum, two academic sub- 
jects through the primary language (Ll). 

Gold is apparently® using category 01 in row 01 of the CDE Language 
Census. The Census instructions state: 

(01) Alternative Course of Study: Classes where EL students are taught 
English and other subjects through bilingual education techniques or other 
generally recognized methodologies permitted by law [emphasis added] and 
where the pupDs enrolled have been (1) granted a parental exception waiver 
pursuant to EC 310 and 311; or (2) enrolled in any Alternative Education 
Program operated under the Superintendent of Public Instructions waiver 
authority (EC 58509) when such an alternative for EL students was estab- 
lished specifically to waive one or more sections of EC 300 through 340; or 
(3) enrolled in a Charter School program which offers any alternative course of 
study [emphasis added] for EL students. 

The problem with the above category is that it includes instructional 
techniques other than bilingual education, such as an alternative course of 
study in a charter school. Most of the time the numbers in the two cate- 
gories are similar, and often they are identical, but for a small percentage of 
the schools they are very different. It seems to me that a category that 
includes only bilingual education is a more appropriate category than one 
that includes any alternative course of study. Moreover, the category that 
includes only bilingual education is the one that has been used by CDE for 
decades and so enables us to compare pre and post Proposition 227 trends. 

If we look down the list of schools in Appendix 1, we can see that about 
a dozen of the so-called successful bilingual schools had about one- third or 
less of their English Learners enrolled in bilingual education using my cal- 
culations and about nine had one- third or less enrolled in bilingual educa- 
tion using Gold^s calculations. None of the schools in either column had all 
of their English Learners enrolled in bilingual education, and across all 
schools only 55 percent of English Learners were enrolled in bilingual edu- 
cation. 

In short. Gold overestimates the percentage of English Learners enrolled 
in bilingual education because he uses the less appropriate CDE category, 
but even his estimate — 62 percent — is low. The situation gets worse when 
we examine the groups whose achievement Gold actually did attribute to 
bilingual education. As shown in Figure 5, across all of Gold’s schools, only 
33 percent of all students, and 44 percent of all Hispanic students, are 
enrolled in bilingual education in these so-called “bilingual” schools. 

Moreover, not all the students were tested, although this varies consider- 
ably from school to school. If we go down the list of schools in column 12 



’He does not say which category he is using nor does he present the number enrolled in bilin- 
gual education (only the percentage is presented). 
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of Appendix 1, we can see that in a number of them one-fourth to one- 
third of the bilingual education students might not have been tested/® In 
one school, Gates Elementary in the Saddleback District, there is no 
English Learner achievement data at all and in another school, Del Rey 
Woods in the Monterey Peninsula School District, there is no bilingual 
education enrollment/^ 

When we examine individual schools, we can also see evidence that the 
testing rates are inflated. Eight schools have more LEP students tested than 
enrolled in those grades and two have so many more LEP students tested 
than enrolled that the data lack credulity. Without adjusting for the inflat- 
ed testing rates, across all schools in Gold’s sample, an estimated 15 percent 
of the bilingual education enrollment might not have been tested. 

The final column in Appendix 1 shows the estimated percentage of 
Hispanic students enrolled in bilingual education who also have test scores, 
adjusting only for the LEP non-testing rate.“ As shovm in Figure 6, the 
percentage of all students enrolled in bilingual education who also have test 
scores is estimated to be 30 percent and the percentage of Hispanic students 
enrolled in bilingual education who also have test scores is estimated to be 
37 percent. These are probably conservative estimates. The actual percent- 
age with test scores would be lower than that shown if we had Spring, not 
Fall, enrollment and if I had adjusted for the non-LEP students who were 
not tested. 

In short. Gold was willing to attribute the school’s achievement growth 
and Hispanic achievement growth to a bilingual program that enrolled only 
one-third of the students and less than half of the Hispanic students, and 
where it is possible that even fewer actually had test scores. 

Conclusions 

Gold is not alone in his wiUingness to rush to judgement without taking 
into account the limitations of the CDE data and the problem of differen- 
tial testing rates between programs and over time. Most of the conclusions 



'®This is estimated by first calculating the number of EL students who were not tested, then 
assuming that all of the non-tested English Learners are enrolled in bilingual education in 
the tested grades. This estimate may seem to be too high, but it is offset by the fact that the 
testing rate is inflated to begin with and we know that it is possible for the individual testing 
rate in bilingual education to be almost half that of English instruction programs, even when 
the test is required. 

"Although the CDE database shows 178 English Learners enrolled in an alternative course 
of study (row 1 of the Language Census) at Del Rey Woods, the row 2 variable shows they 
are being taught in English with some primary language support, not in bilingual education. 

’^This is calculated by assuming that all the bilingual education students in the tested grades 
are Hispanic, subtracting the number of English Learners not tested from that number, and 
then dividing by the number of Hispanic students in the tested grades. 



READ PERSPECTIVES 

158 





drawn in the popular press from simple descriptive analyses of the achieve- 
ment gains of schools, and school districts, are invalid, even when LEP 
achievement is the outcome variable. 

The limitations of the CDE database, typically ignored, are 1) the 
achievement data are only available at the school and school district level; 
2) the test scores are not broken down by program or ethnicity, so it is not 
possible to compare the test scores of English Learners in bilingual educa- 
tion to those in all-English instruction in the same school; 3) the tests are 
taken in grades 2-11, but the CDE does not keep program enrollment by 
grade;^^ 4) the testing rates are inflated by the use of enrollment counts from 
an earlier point in time than the test scores; 5) testing rates vary consider- 
ably from school to school and district to district, and entities with differ- 
ent testing rates cannot be compared to each other unless their testing rates 
are controlled for; 6) the testing rates are higher for all-English programs 
than for bilingual education programs, but it is not possible to adjust pre- 
cisely for this with, the CDE aggregate data; 7) testing rates have gone up 
since Proposition 227, which would obscure achievement gains that might 
be detectable at the aggregate level; and 8) there are two separate sets of 
program categories in the state database with different enrollment counts, 
and one cannot be sure which category a researcher is using. 

Gold (2000), like others, has only muddied the waters by ignoring these 
limitations. It is unwarranted to conclude that bilingual education has had 
a salutary effect on school and Hispanic achievement when only one-third 
of the students and less than half of the Hispanic students were enrolled in 
it and where it is possible that only 30 percent of all students and a little 
more than a third of the Hispanic students were both enrolled in bilingual 
education and had test scores. Although the CDE database is an extraordi- 
nary resource for researchers, and the entire web site is nothing less than 
dazzling, the data themselves have serious limitations and one must be cau- 
tious in drawing conclusions from them. 



'^So if, for example, one wanted to compare the achievement of a school with all of its English 
Learners enrolled in bilingual education to that of another school with all of its English 
Learners in all-English instruction, one would still have to estimate the enrollment in these 
programs for the tested grades in order to control for testing rates. 

‘^Although one set has been used for decades and the other set began with the implementa- 
tion of Proposition 277, it is possible that a researcher might use one set for the pre-1998 
trend and another for the post- 1998 trend. This, of course, would be a mistake, but it would 
not be surprising to find it has been done. The researchers who are analyzing only post-1998 
outcomes, such as Gold, typically use one category rather than another without identifying 
which one they are using. 
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1997-93 (Pre) 



1998-99 (Post) 



1999-00 (Post) 



Figure! 

Percentage of English Learners Tested by Program 
Using Individual Student Date from Los Angles Unified School 
District and Pasadena Unified School District 



93% 




Los Angeles/ 1996-97 Pasadena/* 1997-98 



•Source: LAUSD, 1998. 
••Source: Bali, 2000. 




ROSSELL 

I6f 




Pei 

Califo 



S 



1 997-98 (Pre) 1 998-99 (Post) 1 999-00 (Post) 

L_l No Bilingual Bilingual> 120 Bilingual>240 and % Bilingual>40 

Figure 4 

Reduction in the Percentage of English Learners Who Are Tested with 
Each Additional English Learner Enrolled in Bilingual Education 
Controlling for Other Variables in California Elementary Schools, 
1997-98 through 1999-2000 
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Figures 

Bilingual Exlucation Enrollment 
in Gold’s Successful Bilingual Program Schools, 
1999-00 



44% 



33 % 



All Students 

^Maximum possible. 



Hispanic Students 



Figure 6 

Possible Percentage Enrolled in Bilingual Education Who Also Have 
Test Scores in Gold’s Successful Bilingual Program Schools, 

1999-00 
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Table 1 

Predictors of the Percentage of Eng^sh Learners Tested on the SAT9 
in California Elementary Schools Enrolling English Learners 
in Tested Grades, 1997-98, 1998-99, 1999-00 



1997-98 


Dependent Variables 


Mean 


b 


Beta 


t 


Signif Level 


% of English Learners Tested 
Constant 


76 


64.943 




53.59 


0.000* 


Number Enrolled in 
Bilingual Education 


75 


4D.033 


0.16 


-5.41 


0.000* 


Number of EL who are 
Spanish Speakers 


159 


0.013 


0.10 


3.03 


0.002* 


Total School Enrollment 


617 


0.021 


0.19 


10.22 


0.000* 


% of School on AFDC 
or Calworks 


20 


-0.079 


0.05 


-3.01 


0.003* 


Adjusted r2 




0.035 








N 




4,840 








1998-99 


Dependent Variables 


Mean 




Bela 


t 


Signif Level 


% of English Leomers Tested 
Constant 


82 


70.100 




68.79 


0.000* 


Number Enrolled in 
Bilingual Education 


30 


-0.038 


0.13 


-8.08 


0.000* 


Number of EL who are 
Spanish Speakers 


162 


0.002 


0.02 


0.87 


0.384 


Total School Enrollment 


615 


0.020 


0.21 


11.29 


0.000* 


% of School on AFDC 
or Calworks 


17 


0.019 


0.01 


0.81 


0.420 


Adjusted r2 




0.051 








N 




4,905 








1999-00 


Dependent Variobles 


Mean 




Beta 


t 


Signif Level 


% of English Learners Tested 
Constant 


86 


78.628 




86.77 


0.000* 


Number Enrolled in 
Bilingual Education 


31 


-0.030 


0.12 


-7.39 


0.000* 


Number of EL who are 
Spanish Speakers 


168 


0.006 


0.06 


2.66 


0.008* 


Total School Enrollment 


621 


0.010 


0.12 


6.46 


0.000* 


% of School on AFDC 
or Calworks 


17 


-0.061 


0.04 


2.92 


0.004* 


Adjusted r2 




0.031 








N 




4,862 








* SfatisHcdly significant at .05 or better. 
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Seven Successful 
Bilingual Schools 
In Texas 



John R. Correiro 

T he report titled The Texas Successful Schools Study: Quality Educa- 
tion for Limited English Proficient Students is an important contri- 
bution to the continuing debate over the efficacy of bilingual 
education in the United States. 

As the nation surges forward on school improvement, high stakes test- 
ing, and a “no child left behind agenda,” supporters of bilingual education 
have expressed alarm that Limited English Proficient (LEP) students may 
be allowed to fall through the cracks. 

This report is refreshing to say the least, as it takes a measured, dispas- 
sionate, and scholarly look at a topic that arouses great passion among its 
opponents as well as supporters and, indeed, it was a collaborative effort of 
the Texas Education Agency (TEA), the Charles A. Dana Center at the 
University of Texas in Austin, Texas, the Texas A&M University-Corpus 
Christi, and the seven elementary school campuses comprising the study 
cohort (p, 3). The seven were: Pharr-San Juan- Alamo ISD (“Independent 
School District”); Campestre Elementary, Socorro ISD; Castaneda Ele- 
mentary, Brownsville ISD; Kelly Elementary, Hidalgo ISD; La Encantada 
Elementary, San Benito ISD; and Scott Elementary, Roma ISD. 

The principal investigator of the study was the Program Evaluation Unit 
in the Office for the Education of Special Populations at the TEA. The 
study was thorough — conducted over a 24-month period beginning in 
March of 1998 and ending in March of 2000. 

Specific research questions focused on demographics, and on effective 
practices and characteristics of the seven study sites and the educational 
personnel assigned to the LEP populations. 

Multiple methodologies were employed in profiling the study sites. 
These included: teacher questionnaires; interviews of teachers, campus 
principals, district administrators, and parents; on-site campus and class- 
room visits; and analyses of the Texas Assessment of Academic Skills 
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(TAAS) and Professional Development and Appraisal System (PDAS). 

The report is extensive. It is divided into seven sections: 1) The Executive 
Summary, 2) Introduction and Background, 3) Need for the Study, 4) Find- 
ings, 5) Student and Campus Performance, 6) Appendices, and 7) Referen- 
ces for Further Reading. 

This study is also important because it examines the variables that con- 
tribute to academic success of economically disadvantaged and language 
minority students and because historically school districts, owing to finan- 
cial concerns, relied on national research studies to learn about best prac- 
tices. Here we have a state department of education, state universities, and 
local school districts collaborating on a major research effort to demonstrate 
the results of the implementation of high standards and expectations. 

Furthermore, the study does not try to answer whether the program 
works or is effective, but rather, focuses on challenges of providing appro- 
priate schooling for a growingly diverse population and profiles the contri- 
butions of programs, policies, and school personnel to the academic success 
of the LEP student population. 

The study finds its roots in the Successful Schools Study of the Charles A. 
Dana Center at the University of Texas at Austin and the TEA, which was 
released in February of 1998. That study focused on Title I high achieving 
schools with high poverty rates where at least 60 percent of the students 
qualified for free or reduced price lunches. There were 26 schools in that 
study and the emphasis was on identifying good practices that enable high- 
poverty schools to create an environment in which the majority of students 
achieve high levels of attainment on the TAAS (p. 10). 

The seven schools of this study were drawn from the original 26 and met 
an additional set of criteria requiring that: 

■ Schools enrolled more than 40 percent LEP students during the 1996- 
97 school year 

■ Schools enrolled more than 50 percent economically disadvantaged stu- 
dents during the 1996-97 school year 

■ Schools had zero TAAS exemptions during the 1996-97 school year 

■ Schools met the criteria for a rating of either “Recognized” or 
“Exemplar” in the Texas school accountability system based on the 
Academic Excellence Indicator System (AEIS) of May 1997 that 
included English TAAS scores in reading, writing, mathematics, and 
attendance rates 

What we see in this study, then, is a focus on the academic success of 
LEP students in seven elementary schools that met a rigorous review for 
inclusion. 
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Section III, The Need For the Studyy chronicles the growing number of 
LEP students across the state of Texas — a phenomenon that urban areas 
and school districts face from coast to coast. 

In a four-year period — 1993-94 to 1997-98 — the total state enrollment 
in Texas Public Schools grew by a total of 290,038 students, from 3,601,834 
to 3,891,877. According to Table 16, in Appendix B to the report (prepared 
by the TEA Public Information Management System), of the 290,038 new 
students, 257,706 were comprised of ethnic minorities as follows: 

■ Hispanics — 195,564 or 67 percent 

■ African-American— 44,999 or 16 percent 

■ Asian/Native- American — 17,143 or 6 percent 

The challenges posed by this dramatic increase were both obvious and 
daunting, especially when one considers that this growth in the LEP pop- 
ulation represented a 44 percent increase for the six-year period between 
1991-92 and 1997-98 (p.l5). 

These challenges then, not unUke those faced by urban school districts 
across the nation, included teacher shortages and the need for specific pro- 
grams to address the education of differing linguistic groups. Further com- 
pounding the Texas picture in 1997-98 was the fact that 77 percent-of the 
LEP population was enrolled in elementary grades, exacerbating the 
demand for specialized teachers with required biUngual certification to 
meet the needs of this population (p. 16). 

In comparing the growth of teachers during the same period of student 
enrollment growdi, at first glance (Tables 19 and 20, Appendix -B) it 
appears that new hires were adequate to meet student needs. A closer look, 
particularly at the students’ programs and grade spans, reveals a fuzzy pic- 
ture. Of the new teachers assigned to non-bilingual grades 1-6, 95 percent 
were certified for their positions, while of the new teachers assigned to 
bilingual grades 1-6, only 59 percent were certified for their positions. 

The need for this study became apparent as the TEA sought to provide 
focused leadership to assist school administrators and policy-makers in 
their efforts to meet the new challenges. Thus, the TEA saw this study as 
one in a needed series of studies aimed at providing guidance to school 
boards, administrators, principals, and teacher training institutions to initi- 
ate new programs as well as to expand current ones. 

While the findings of this study are impressive indeed, the authors point 
out that it is one in a series of studies that is needed to provide administra- 
tors and teachers with the necessary guidance and information to provide 
for the educational needs of language minority students. 

Findings were detailed in three specific areas: Findings Related to Effective 
School CorrelateSy Findings Related to Other Literaturey and Findings Related 
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to Research Questions, 

In responding to one of the research questions, “What is the relationship 
between campus practices and theory?,” the research team reviewed the lit- 
erature on the education of language minority students, and the first set of 
findings reflects the results of the review in relationship to the study find- 
ings. Literature on effective schools research was prominent, especially the 
“Seven Effective School Correlates” (p. 19). 

The importance of the Seven Effective Schools Correlates for this study 
is that they have been used in school improvement and equitable education 
for all children (Effective Schools Correlates, 1998) and appeared promi- 
nendy in the analysis of the bilingual programs in the seven successful 
schools. 

The Effective Schools Correlates are: 

1) Clear School Mission 

2) High Expectations for Success 

3) Instructional Leadership 

4) Frequent Monitoring of Student Program 

5) Opportunity to Learn and Student Time on Task 

6) Safe and Orderly Environment and 

7) Home- School Relations. 

Not surprisingly, the seven successful schools evidenced most of the cor- 
relates, although researchers observed different dynamics and added 
dimensions related to bilingual students. These dynamics helped to trans- 
form the schools from effective to “Exemplary” according to the TEA 
Academic Excellence Indicator System (p. 19). (Detailed information on 
how the correlates were addressed in the seven successful schools, can be 
found in the individual case studies beginning on page 36 of the report.) 

In all of the seven successful schools, there was a clearly articulated mis- 
sion, indicating that for bilingual students there were clear instructional 
goals linked to academic achievement. There were assessment and account- 
ability procedures linked to benchmarks. Immediate, consistent, and multi- 
ple remediation strategies were available to students not making the bench- 
marks, and these were provided before, during, and after school. The school 
community was aware of the mission statements, thereby enabling the 
stakeholders to demonstrate total commitment to fulfilling the mission of 
the school (p. 19). 

Again, in aU seven of the successful schools, staff expected students to 
perform at a high academic level. Stakeholders shared that view ands 
believed that they could empower students to succeed (p. 20). 

In one school, where primary grade teachers finished their day a half- 
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hour earlier than the upper grade classes, these teachers stayed on to pro- 
vide tutoring and additional help to the upper grade bilingual students. 
Further, teachers in this school shared materials and strategies and planned 
vertically and horizontally across grade levels as w^ell. Both English and the 
students’ native language wctq used to affirm student success and to provide 
encouragement. Students flourished in this environment, exhibiting a belief 
in their own capacities to succeed. 

The principal as an instructional leader is another of the effective schools 
correlates that was evidenced in the study of the seven successful schools. It 
is important to note that the principals were all experienced, certified bilin- 
gual teachers before becoming principals and aU had prior grounding in the 
principles of bilingual education both from an educational and an experi- 
ential base. They were able then to provide support and guidance to their 
teachers in effective bilingual practices. 

Principals in the seven successful schools were able to communicate 
effectively to the faculty, students, and parents and to all stakeholders, con- 
tributing significantly to the cohesiveness of the school community. 

Effective schools monitor student progress frequently. This was mirrored 
in the seven successful schools, and student progress was measured in both 
English and the native language. Monitoring of students’ progress in their 
native language was important in these schools since development in the 
native language was considered essential to success in English. 

Time on task is another characteristic of the effective schools correlates, 
and it was clear to the researchers that the teachers at the seven successful 
schools structured time appropriately for dual language instruction. 
Multiple strategies were employed — taking into account different learning 
styles — such as small and large group instruction, cooperative learning, and 
instructional technology use. The seven successful schools were character- 
ized by a high degree of time on task. TAAS strategies were emphasized 
and again both English and the native language were mediums of instruc- 
tion, depending upon the LEP students’ placement (p. 21). 

The seven successful schools were demonstrative of a “family atmos- 
phere” which contributed greatly to making these schools “effective” (p. 21). 
The community/school relationship is key, to “ownership” of the schools 
where the stakeholders see the schools as “ours.” Parents felt strongly about 
and articulated that an “open door” policy permeated the schools. Students 
were happy; parents utilized the schools for a variety of activities, including 
recreation; vandalism was down; gang activity was reduced and students, 
consequently, felt safe at the schools. 

Home and school relations depend upon the parents understanding the 
mission of the school and on their subsequent involvement in the school 
community. The seven successful schools had exceptional parental support 
and it was evident to the researchers that parents took great pride in their 
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schools, so much so that the limited use of English by parents was not seen 
as a barrier to their participation. Researchers suggested that the value in 
which the community culture was held by administrators and staff gave rise 
to the strong response of parents. 

What is truly impressive about these findings is that the use of the stu- 
dents’ native language is important in these schools. The native language is 
utilized as an effective and necessary teaching tool to assist in the transfer 
of knowledge to English. It is measured, supported, and valued. Adminis- 
trators, teachers, students, and parents are not afraid to use the native lan- 
guage. In fact it becomes the bridge across which knowledge is transferred 
into English. Native language use is accomplished within the framework of 
the schools’ clearly articulated educational mission. In these schools, the 
native language of the community is valued and celebrated. 

The literature review also identified another school correlate — collabora- 
tive leadership. Collaborative leadership is not a new concept. Researchers 
at the Northeast and Islands Regional Educational Lab at Brown Univer- 
sity point out that several approaches have been promoted in the educa- 
tional literature over the last decade (Proposal, p. 144). The Lab researchers 
point out that the Council of Chief State School Officers (CCSSO) is call- 
ing for the development of collaborative leadership models (CCSSO, 
2000a). As CCSSO notes, “Student, teacher, parent and school board lead- 
ers, along with leaders of voluntary associations, community groups, and 
faith institutions, have important responsibilities to work together vnth ex- 
ecutive leaders to create and sustain effective schools” (CCSSO, 2000a, 7). 

In his paper prepared for the Albert Shanker Institute, Building a New 
Structure for School Leadership (2000), Richard Elmore of Harvard Univer- 
sity calls for leadership based upon distributed leadership models. He lists 
five principles as a framework: 

1) The purpose of leadership is the improvement of practice and perform- 
ance, regardless of role. 

2) Instructional improvement requires continuous learning. Learning is 
both an individual and social activity, and collective learning involves the 
creation of a strong normative environment to guide and direct the 
acquisition of new knowledge. 

3) Learning also requires modeling. Leaders must lead by modeling the 
value of behaviors that represent collective goods. 

4) The roles and activities of leadership flow from the expertise required for 
learning and improvement, not from formal dictates of the institution. 

5) The exercise of authority requires reciprocity of accountability and capa- 
city. 

(Elmore, 2000, p. 20-21) 
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In the seven successful schools, one could identify various aspects of the 
Elmore construct at work. Further, in these schools, collaboration was syn- 
onymous with empowerment. Administrators and teachers were entwined 
in the decision-making around curriculum, principals provided needed 
materials and resources enabling teachers to accomplish their goals, com- 
munication flowed up and down the professional hierarchy and across the 
home/school community effectively, and principals provided teachers with 
adequate planning time and teachers used the planning time effectively, 
giving constant feedback on students to principals, parents, students, and 
each other. 

Besides the Effective Schools literature, the review also focused on the 
important characteristics of school reform. These included: 

1) Distributive Leadership 

2) Campus Leadership 

3) Teaching Staff 

4) Teaching Practices 

5) Parents’ Role and 

6) Program Characteristics. 

Here the findings of the study show that district leadership is important to 
the success of LEP students. In the seven study sites, district support for 
administrators and teachers “includes regular professional development” (p. 
22 ). 

In the Socorro ISD, additional staff members worked with teachers of 
LEP students in school settings (p. 22); and in Pharr- San Juan- Alamo ISD, 
summer institutes provided staff development for LEP teachers. In both 
districts the bilingual coordinator was responsible for the programs. 

What is more, appropriate funding at all seven sites was assured as a con- 
sequence of district leadership. In looking at campus leadership, the study 
reports, “Each of the seven principals of the successful schools has a 
Master’s degree and extensive training in and certification in bilingual edu- 
cation and ESL...[and] had also taught LEP students for no less than five 
years, thus having knowledge of bilingual education philosophy and theo- 
ry (p. 22). 

The literature emphasizes the importance of certification standards to a 
successful bilingual program (p, 22), The study is emphatic in describing 
the role of all seven principals as “instructional leaders.” Teachers in the 
seven schools described the principals as “collaborative,” all had “high 
expectations,” and all exhibited behaviors of “empowerment and trust” in 
relationship to the teachers (p. 23). 

Principals were also advocates of professional development regarding 
dual language literacy development, acquisition of second language theory. 
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TAAS strategies, and curriculum integration (p. 23). The principals kept 
abreast of test scores and because of their content experience and expertise 
were able to look at LEP students’ needs from a broad perspective and pro- 
vide critical guidance and support to teachers. Principals were thoroughly 
immersed in school/community relations and parents were most apprecia- 
tive, feeling welcome in the seven schools (p. 23). In essence, principals in 
the successful schools “walked the talk” of research literature on “instruc- 
tional leadership.” 

Findings relative to the teaching staffs in the successful schools mirror 
what the research literature emphasizes: 

■ “All teachers assigned to the LEP populations... are bilingual or ESL 
certified” (p. 24). This is essential to long term success in the programs 
(Hakuta et al., 1997). 

■ “...[T]eachers are committed to achieving equity for all students and 
believe that they are capable of making a difference in their students’ 
learning” (Saravia-Shore and Garcia, 1995). Most of the staff at the 
seven schools have been in their teaching positions for more than ten 
years and they attribute this longevity and stability as contributing to the 
success of the students (p. 24), 

■ Carter and Maestras (1982) defined a successful school climate as one 
that includes: 

1) High staff expectations for students of the program, 

2) Strong demands for academic performance, and 

3) High staff morale. 

The study shows clearly that “teachers believe that aU students can learn 
and have high expectations” (p. 24). 

Findings in the area of teaching practices are rich with multiple and suc- 
cessful research-based strategies employed by the teachers at the successful 
schools. One of the most salient findings in the area of teaching practices 
was the use of Spanish and English for direct instruction in all classrooms. 
Hakuta in 1997 pointed out that “use of the home language is necessary for 
success with second language learners and does not impede progress in 
English.” 

The study tells us that Willig (1985) and Wong-FUlmore and Valadez 
(1986), the former using meta-analysis to combine academic achievement 
scores from a large set of statistically unrelated studies, and the latter con- 
ducting a more traditional review of related independent studies, reached 
the same conclusion: “that bilingual programs significandy enhanced aca- 
demic achievement, in comparison to English instructional programs” (p. 
25). Further, “teachers acknowledging equal prestige to both English and 
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Spanish language,.. is an essential characteristic of success (Carter and 
Maestras, 1982 and Hakuta et al., 1997), 

Consequently, based upon the research, success could have been predict- 
ed at the seven schools, i.e.: 

■ At all seven schools the use of Spanish and English is evident in all class- 
rooms (p, 24). 

■ State adopted materials, and other resources, were available in classrooms 
in both English and Spanish for use in instructional activities as needed. 

■ Students were affirmed by their teachers for their response in either lan- 
guage. 

Teachers eschewed the use of ditto worksheets in favor of focused in- 
struction in small groups, paired groups and cooperative groups or skills- 
focused groups suggested by the research findings of Garcia (1994), Kagan 
(1989), and Tinajero et al. (1993). AU the aforementioned have noted the 
importance of cooperative learning practices for Latinos and language 
minority students of different backgrounds. 

Garcia (1988) points out that an integrated curriculum, responsive to the 
linguistic ability of students and taught by trained bilingual/biliterate 
teachers, was common where students’ high standardized achievement test 
scores were above national norms and where students’ bilingualism was 
affirmed as a matter of pride. Five of the seven successful schools in partic- 
ular — Castaneda, Scott, Campestre, Bowie, and Clover — through collabo- 
ration and planning developed integrated instructional units that have been 
very successful with LEP students. These units are constantly reviewed and 
updated and address the TAAS skills and provide test taking practice for 
the LEP students (p. 25). 

Early childhood teachers at three of the sites — Castaneda, Bowie, and 
Clover — ^were trained in and implemented Montessori strategies — encour- 
aging students to work independently. According to the study, Quintero in 
1988 pointed out that pre-school programs that support child-centered 
independent learning centers lead to success in pre-school for Latino chil- 
dren. 

The findings on teaching practices in the successful schools also refers to 
the research of Roser et al. (1989) and Tinajero et al. (1998) that suggests 
children will become successful in listening, speaking, reading, and writing 
when teachers surround children with literature and provide time to engage 
in the language arts. Again, the study demonstrates that literature-based 
integrated units are used in many of the classrooms of the successful 
schools. 

Math and science centers were observed in many of the classrooms and 
the use of manipulative and hands-on activities are common practice in the 
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teaching of math and science, the success of which is supported by De La 
Cruz, 1998, who noted, “[T]o ensure that instruction is at a high level 
where every student can experience success, manipulatives can be used to 
demonstrate a concept so that new information can be processed.” 

Student progress is monitored closely and students receive considerable 
support: after- school tutoring, special computer-assisted instruction, foc- 
used literacy development, Reading Recovery, and re-teaching by classroom 
teachers. The research of Hakuta et al., 1997, states that “continual and reg- 
ular monitoring assures success for students...” (p. 26). 

The study indicates that parents have high expectations for their children 
as well as respect and high regard for teachers and principals (p. 26). At the 
successful schools, parents were engaged in all aspects of their childrens 
learning from homework to volunteering in the school to participation in 
the schools’ parent centers, Gonzalez research in 1998 undergirds the 
report’s findings on the role of parents: “The role of parents is critical in the 
education of the second language learner as the family adjusts to the cul- 
tural and linguistic demands of the community and school” (p. 21). 

The study documents the explicit characteristics of the successful 
schools: 

1) Administrators and teachers are focused on the development of pro- 
grams that will meet the needs of the LEP students. 

2) Regardless of the bilingual program model, administrators and teachers 
were committed to implementing the program based upon solid research 
and state guidelines to educate LEP students. 

3) Teachers emphasized use of the home language as a medium of instruc- 
tion and developing proficiency in each of two languages. 

4) Literacy in the first language of minority language speakers is developed. 

5) There is strong focus on the delivery of curriculum. 

These characteristics stood out among the keys to success in the seven 
schools because of their foundation in solid research. Teachers focused on 
curriculum objectives, using planning time well, and principals provided 
adequate time for teachers to meet by scheduling students in non-core cur- 
ricular subjects. Multiple instructional strategies were utilized — direct in- 
struction to the large group, and teacher aid instruction provided to small 
groups, especially at primary grade levels. Professional development, in- 
cluding language acquisition training and attendance at conferences and 
workshops provided by the state, by area institutions of higher education, 
and by independent consultants, was an important ingredient to success. 
After-school tutoring that included enrichment activities and accelerated 
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reading programs were also important support systems available in the suc- 
cessful schools. 

There were six research questions that the Successful Schools Study addres- 
sed in the area of demographics, effective practices, and characteristics of 
the seven sites. These research questions and their findings are as follows: 



1) What are the district leadership practices that facilitate academic and linguis- 
tic growth/success for language minority students^ 

■ Staff development focusing on language acquisition, bilingual method- 
ologies, and TAAS objectives 

■ Workshops on teacher expectations and student achievement and John- 
son and Johnson cooperative learning 

■ District practice of engaging bilingual experts to conduct professional 
development activities 

■ Adequate campus budgets for bilingual programs 

■ Acquisition of technology — hardware and software 

■ District bilingual administrators provided guidance and oversight for 
program implementation 

■ Teachers and administrators were guided by district policy documents 
that detailed the plan of implementation to educate LEP students 

■ Identification, assessment, and placement procedures for LEP students 
and the roles and responsibility of the Language Proficiency Assessment 
Committee (LPAC) were clearly spelled out. 

2) What are the campus leadership practices that facilitate academic and linguis- 
tic growth/success for language minority students^ 

■ Communication with parents of LEP students that conveyed and fos- 
tered a caring and positive attitude for bilingual students 

■ Principals held high expectations for all students and insisted on linguis- 
tic development of the LEP students 

■ The belief that extensive training and certification in bilingual education 
of the principals was a major factor in focusing on opportunities for and 
attention to the LEP students 

■ Principals empowered teachers to make instructional decisions, strength- 
ening the notion of shared leadership 
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■ Use of the home language (Spanish) to develop a mastery in the language 
before transitioning LEP students into all English classes 

■ Principals v/ere true partners of the instructional team. 

3) What are the characteristics of the teaching staff that facilitate academic and 

linguistic grovjth/success for language minority students^ 

■ Total commitment and dedication by the bilingual teachers assigned to 
work with the LEP students 

■ All teachers of the LEP students were certified in bilingual education or 
ESL 

■ All teachers at the seven sites indicated extensive participation in profes- 
sional development around bilingual education and second language 
acquisition through district, state, and conference offerings 

■ Ethnicity and background of teachers assigned to the LEP students pro- 
vided appropriate role models and bilingually proficient teachers at every 
grade level 

■ Excellent teaching at each of the seven sites. 

4) What are the effective teaching practices that facilitate academic and linguis- 
tic growth/success for language minority students'^ 

■ While no one specific model of bilingual education was implemented at 
all study sites, effective teaching was the result of instructional focus and 
curriculum adaptations that were aligned vnth the linguistic and aca- 
demic levels of LEP students, using dual language as mediums of 
instruction 

■ Teachers allow and encourage LEP students, who were more proficient 
in Spanish, to respond to instructional cues in their home language 

■ In the late-exit model, the home language was used until there was evi- 
dence of literacy in both languages. Teachers determined literacy when 
students exhibited academic success in both languages 

■ Other successful strategies at the seven sites included early grades whole 
language strategies, literacy-rich classroom environments, phonics devel- 
opment, and music used to reinforce oral language development 

■ Stories in English on audio-tapes sent home for children and parents to 
practice contributing positively to the development of English language 
for both students and parents 

■ The teacher’s ability to diagnose when the use of one or the other lan- 
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guage was most appropriate with students, regardless of grade level(s) 

■ Grouping for instructional purposes (English/Spanish) ensures that 
instructional focus is appropriate for the language level and the academ- 
ic level, contributing greatly to linguistic and academic development of 
LEP students, since not all students are treated with the same instruc- 
tion. 

5) What are the characteristics of parents and parental involvement of the seven 

campuses^ 

■ Parents supportive of teachers and programs for LEP students and 
encourage their children in school and at home 

■ Parents are positive models for their children by participating in school 
and parent center activities and by being visibly connected to the schools 
and their children 

■ Parents assist students by volunteering to read and serve as tutors and 
ethnic story tellers in the classroom 

■ Parents believe that teacher/parent communication — using a variety of 
means — was consistent whether the communication was positive or neg- 
ative 

■ All communication with the parents was in English and Spanish, de- 
pending upon the language proficiency of the parents 

■ Instruction was provided to parents to help improve their English skills. 

6) What are the characteristics of program(s) serving language minority stu- 
dents? 

■ Principals’ and teachers’ program knowledge regarding the education of 
language minority children 

■ Spanish and English were used as languages of instruction, ensuring lan- 
guage development of the weaker language and literacy development of 
the stronger or dominant language 

■ Principals, teachers, and parents afforded equal prestige to Spanish and 
English in the classroom, a practice that can be a factor instilling pride 
and encouragement for LEP students and parents to want to succeed 

■ Implementation of effective reading practices that support the goal of the 
“Texas Reading Initiative” 

■ Early data collection on LEP students allows for informed and appro- 
priate decisions regarding students’ instructional needs 
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■ Coordination of and opportunity to participate in a host of special pro- 
grams as enhancements to the regular program 

■ Students are afforded equal opportunities to quality education and an 
appropriate opportunity to equal benefits from programs and services 
offered 

■ Budgetary supports — from a variety of sources, i.e., district, state, and 
federal grants — result in program enhancements or value-added charac- 
teristics that impact positively on the language and academic needs of 
the LEP students. 

There is a wealth of information in the Texas Successful Schools Study, 
enough to provide scholars and students of bilingual education with rich 
and informative reading. More than 50 pages are devoted to an analysis of 
student performance at each of the selected school sites and another 36 
pages compare the data of LEP, former LEP, and never LEP students to an 
external campus group and to the TEA comparison campus groups as 
measured by the TAAS from 1994-95 in English and Spanish, as appro- 
priate (p. 4). 

Section V, Student and Campus Performance on pages 91-128, presents 
a performance analysis conducted for the campuses in the study and com- 
pares these schools to external cohorts. There are 15 tables and 15 graphs 
that provide data on percentage of students passing TAAS reading and 
math, as well as depicting similar data on the Texas Learning Index (TLI), 
and graphs that provide data on pre/post oral langage assessment categories 
for LEP students. Data are reported by grade level. 

This section also outlines the caveats to be considered when interpreting 
the data and clearly underscores the complexity of the study, while offering 
considerable optimistic information on LEP student progress on the 
TAAS. The author of this review recommends that interested scholars and 
specialists in psychometrics will find the data on pages 91-128 as well as 
pages 167-182 of greatest use. The individual school reports are replete 
with school profiles, staff and budgetary information, instructional and 
implementation practices, and teacher questionnaire results. The 36 pages 
of comparative data and analysis are supported by extensive charts and 
graphs. (Complete tables, charts, etc. are available on request from the 
READ Institute offices in Sterling, Virginia.) 

The summary of findings in the Student and Campus Performance sec- 
tion of the report offers the possibility for the development of further 
research and inquiry into what is a very promising picture for LEP students. 
What one can glean from this report is that success for LEP students in an 
era of high expectations, high standards, and high stakes testing is within 
their grasp, provided the political and financial will of the legislatures, state 
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departments of education, and local school districts are committed to assur- 
ing success for all students. 

As this study shows, programs rooted in proven education research that 
calls for a clear school mission with high expectations, driven by instruc- 
tional leadership that assures student progress is monitored frequently, with 
sufficient time on task for both students and teachers, in a safe and orderly 
environment, buttressed by effective home and school relations, is a recipe 
for success. But that recipe requires significant districtwide support in the 
way of funding to provide the consistent professional development related 
specifically to the schools’ programs and the provision of adequate resourc- 
es, equipment, and facilities. 

The Texas Successful Schools Study is encouraging and provides the 
framework that can lead to the building of successful programs for LEP 
students everywhere. 
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Meeting the Needs 
of Students with 
Limited English 
Proficiency 

A Critique of GAO's Report 

Jim Littlejohn 

Introduction 

I n 1998 and 1999, Congress held oversight hearings on how to 
improve federal requirements attached to bilingual education monies 
to give greater flexibility to state and local education agencies in 
teaching Limited English Proficient (LEP) students. The hearings 
also examined oversight issues related to the Office for Civil Rights (OCR). 
Several witnesses introduced evidence that since the early 1990s OCR had 
imposed new, expanded requirements for schools enrolling LEP students. 
In the view of the critics, these sub-regulatory (i.e., unpublished) require- 
ments were both arbitrary and prescriptive, affecting almost all aspects of 
school operations, including curriculum, teaching staff, and other programs. 

In late 1999, after congressional committees had reviewed the data and 
testimony from the hearings, Congress asked the General Accounting Of- 
fice (GAO) to conduct an investigation that would answer the following 
questions: 

1. How long do children with limited English proficiency need to become 
proficient in English? 

2. What approaches are used to teach children with limited English profi- 
ciency, and how long do students remain in language assistance pro- 
grams? 
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3. What are the requirements for children vsdth limited English proficiency 
that OCR expects school districts to meet, how are they set forth, and 
what has been the nature of the interactions between OCR and school 
districts in those instances in which OCR has entered into an agreement 
with the school district concerning language assistance programs? 

The GAO report, submitted to Congress in February 2001, does a poor 
job of answering the questions posed by Congress and fails to provide the 
information that body needs to make long-term policy decisions about the 
federal role in supporting programs for LEP students.^ Instead of providing 
a balanced analysis, the GAO report largely parrots back the views of bilin- 
gual advocates, and virtually ignores the views of experts who argue for 
more English Immersion programs. GAO^s treatment of the educational 
issues is replete vsdth footnotes, tables, and charts, but its one-sided ap- 
proach, combined with the numerous errors and omissions of facts, results 
in an unreliable presentation. 

As bad as GAO’s analysis of the educational issues is, the section dealing 
with OCR is worse. In summary, the GAO “analysis” of OCR’s activities in 
enforcing the civil rights laws related to LEP students is nothing more than 
a whitewash. The material below critiques key aspects of the GAO report 
with respect to the three broad categories of congressional interest. 

Time in Program and 
Language Proficiency Issues 

The authors of the GAO report characteristically cite at length the views of 
bilingual education advocates, giving these views substantial credibility, and 
then toss in a footnote indicating that there are researchers who disagree. 
In one of several examples of this practice, GAO reported: 

■ No clear consensus exists on the length of time children v^th limited 
English proficiency need to become proficient in English. 

■ [N]o agreement exists about how proficiency should be defined or meas- 
ured. 

But apparentiy appealing to a higher authority, GAO cites the popular 
view among bilingual advocates (e.g., Virginia Collier and James 
Cummings) that: 

[I]t may take 4 to 8 years to develop the language skills needed to perform 
on a par with native English-speakers in all core academic subject areas 
(reading, language arts, social studies, science, and mathematics). 

In the same paragraph, the report acknowledged, “However, some 
researchers have concluded fewer years are needed.” This addendum and 
accompanying footnote was a passing recognition by GAO of the research 
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of Dr. Christine Rossell and Dr. Keith Baker, who have argued for many 
years that research data show that English Immersion and similar programs 
are more effective in teaching ELL students than are bilingual education 
programs. 

But GAO^s statement about Dr. Rossell and Dr. Baker is embarrassing- 
ly wrong. Neither has ever concluded that fewer than 4 to 8 years are need- 
ed “to develop the language skills needed to perform on a par with native 
English-speakers in all core academic subject areas (reading, language arts, 
social studies, science, and mathematics.” A quote from Dr. Rossell's article 
“Is One Year Enough?” highlights the inaccuracy of the GAO characteri- 
zation of her position: 

What litde research there is suggests that although it could take a decade for 
a student to reach the highest level of English language achievement they are 
capable of — ^with students who come to the U.S. at earlier grades reaching it 
sooner than students who enter in the later grades (Rossell 2000) — ^virtually 
all [ELL] students understand enough English sometime during the first 
year to be able to comprehend English instruction.^ 

Dr. Rossell has specifically pointed out in her writings that there are two 
separate issues: (1) How long does it take LEP students to learn sufficient 
English to participate in an English-speaking classroom?, and (2) How 
long does it take LEP students to “catch up” with their English-speaking 
peers in English language achievement? She has consistendy stated that 
research shows that LEP students can and do learn sufficient English to 
participate in an English-speaking classroom within a relatively short peri- 
od of time (i.e., one year or less). After achieving a sufficient level of Eng- 
lish proficiency to understand the teachers’ instructions in English, these 
students can then learn the required academic materials in English, which 
obviously entails learning new academic terms and concepts along with reg- 
ular English speakers (who presumably don’t know them either). 

Thus, Dr. Rossell generally agrees that it may take several years for many 
LEP students to achieve full English language academic proficiency, but 
disagrees with the claim of bilingual education advocates that teaching 
these students in Spanish or any language other than English for 4-8 years 
has been shown to be necessary or effective in accelerating their academic 
accomplishments. The 4-8 year range has everything to do with how long 
bilingual advocates believe LEP students should be taught in a language 
other than English (i.e., bilingual education classes) and nothing to do vtith 
the amount of time LEP students should receive remedial assistance 
through the English language. 

GAO listed the criteria that led to its selection of only 4 studies — out of 
70 — for analysis. The criteria included: (1) focus on the length of time chil- 
dren need to become proficient in English, (2) reach a specific conclusion 
about the length of time, (3) have English as the second language learned 
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by the students, and (4) involve original research supported by the data. 
(GAO report, p. 34) GAO did not explain why the application of its crite- 
ria yielded, for the most part, researchers who are strong advocates of long- 
term bilingual education programs (e.g, Virginia Collier, Wayne Thomp- 
son, and James Cummings),^ 

Perhaps GAO should have considered one more criterion — the accuracy 
and scientific integrity of the research. In 1998, Dr. Rossell, in her article 
“Mystery on the Bilingual Express,” analyzed in some detail the Collier 
studies on the length of time LEP students should remain in alternative 
language programs and concluded that: 

[T]his [Collier] report consists primarily of theories of bilingual education 
and criticism of the scientific method. The methodology of the study is 
unscientific, as is the case with all of Virginia Collier’s research.'* 

GAO compounded its errors as regards Rossell s and Baker’s views by 
omitting from the report any explanation of their positions. Both are 
thoughtful, competent researchers who have rebutted with hard facts the 
so-called research of bilingual advocates. GAO is the nonpartisan inves- 
tigative arm of Congress and Congress needs to hear the positions of both 
sides in any debate. The errors and omissions in this report, and the shal- 
lowness of the GAO analysis, call into question its nonpartisan status and 
substantially undermine its usefulness to Congress. 

Approaches Used to Teach English Language 
Proficiency and Length of Time Students 
Remain in Such Programs 

GAO described the two overall categories of educational approaches for 
teaching ELL students as: 

(1) An English-based approach that “uses English and makes little use of a 
student’s native language.” GAO noted, “[Pjroponents of an English- 
based approach expect children to learn English fairly quickly, in 2 to 3 
years.” 

(2) A bilingual approach is designed to take much longer teaching English, 
GAO explained that, “While bilingual programs vary in both their 
goals and length, those programs that promote native-language literacy 
as well as English language literacy may take 5 to 7 years to complete.” 

What the GAO report does not provide to Congress is any useful analy- 
sis of the real-life differences between these two approaches — something 
that goes beyond the theories and spin to the practical issues of implemen- 
tation. 

For example, it would have been much more helpful for Congress to have 
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more factual information about key related issues such as: 

■ whether a large number of LEP students who are placed into bilingual 
classes must first be taught to speak and understand their native language 
well before being taught in English; 

■ whether learning to read first in a student’s native language actually (not 
just theoretically) enhances learning to read in English (the 1997 Na- 
tional Research Council Report, Improving Schooling for Language-Min- 
ority Children found it did not); 

■ how large numbers of Asian students, who come to this country speak- 
ing no English, learn English quickly and then excel in schools and col- 
leges in disproportionate numbers; and 

■ the logistical problems inherent in teaching bilingual education pro- 
grams, including: 

□ the acute shortage of bilingual teachers and the corresponding need 
for many large schools to recruit from foreign countries so-called 
bilingual teachers who speak English poorly; 

□ the high level of segregation of LEP students from English-speak- 
ing students over a period of several years; 

□ the burdens placed upon school administrators and bilingual edu- 
cation teachers to develop classroom materials in several languages; 
and 

□ the fact that bilingual education programs delay for years teaching 
English to LEP students and keep them in bilingual education 
programs for years. 

GAO might argue that Congress did not specifically ask for answers to 
these questions, but any responsible analysis would have included such 
information. 

An even more inexplicable omission is GAO’s failure to discuss 
California’s Proposition 227 and its successes to date. Given the publication 
date of the GAO report (February 2001), there was ample time to review 
the widely published success stories and positive data coming out of 
California that showed remarkable and consistent test score gains statewide 
for ELL students. Ken Noonan, the superintendent of Oceanside Schools, 
who had previously supported bilingual education, decided after passage of 
Proposition 227 to wholeheartedly implement English immersion pro- 
grams. A year later, his school district achieved national recognition because 
of the startling improvements in test scores of LEP students. 

It is inexcusable for GAO to ignore the data firom a state that enrolls 40 
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percent of LEP students in the nation. How can such “analysis of data” be 
taken seriously? 

Furthermore, available data on both the types of programs that LEP stu- 
dents are enrolled in and the length of time the students stay in such pro- 
grams are sketchy and unreliable. As stated below, GAO described the dif- 
ficulty in obtaining solid data about the length of time students remain in 
either English-based or bilingual education programs: 

We found no national data on the length of time children with limited 
English proficiency actually spend in programs [aimed] at helping them 
become proficient in English. Thus, we contacted education agencies in 12 
states ... Of the 12 states contacted, 6 had information on the length of time 
children with limited English proficiency spent in language assistance pro- 
grams. [These were Arizona, Florida, Illinois, New Jersey, Texas, and 
Washington] (GAO, 20) 

California, with about 40 percent of the nation’s students with limited 
English proficiency in 1996-97, did not have statevsdde data that could be 
used to determine how long children were spending in its programs. (GAO, 

21) 

Because of the lack of statewide data in California, GAO obtained time- 
in-program data for LEP students from four large California school dis- 
tricts — Los Angeles (5 years or more), San Francisco (5 years or less), Santa 
Ana (5 years or less), and San Diego (7 years or more). GAO appropriate- 
ly cautioned: “Because of the limited number of states and school districts 
from which the data were drawn, these results should be interpreted cau- 
tiously.” But this did not keep GAO from making its own sweeping con- 
clusions, such as the sidebar in large caps and bold type that stated: 

Most Children in the Six States Reviewed Spend 4 Years or Less in 
Programs Aimed at Increasing English Proficiency (GAO, 20) 

One wonders why GAO felt that particular conclusion should be high- 
lighted, when the report contains information that, for example, 41 percent 
of LEP students in Arizona spent more than 5 years in such programs. 
Similarly, the pre-Proposition 227 data from Los Angeles and San Diego 
showed that LEP students remained in bilingual education programs for 5- 
7 years or more — and still could not function in English! 

OCR Requirements and Interactions 
with School Districts 

Instead of analyzing OCR’s requirements and the procedures that are 
actually followed during school investigations, GAO accepted uncritically 
OCR’s statements about what the law requires and its procedures. Informa- 
tion suggesting substantive contradictions between OCR’s procedures and 
practices was routinely ignored. 




READ PERSPECTIVES 

190 




For example, OCR provided GAO a flow chart (GAO, 11) that describes 
OCR’s investigative process for both complaint and compliance review 
investigations. Two major steps in the “Investigation Phase” of the chart are 
clearly presented as “OCR Issues Letter of Findings” followed immediate- 
ly by “OCR and District Sign Negotiated Agreement.” GAO specifically 
stated that in the five cases that GAO reviewed in depth, OCR followed 
the Title VI investigative process listed in the flow chart. (GAO, 10) GAO 
also stated, “If OCR found a school district was notin compliance with the 
civil rights laws, it worked with the district to negotiate an agreement . . .” 
(GAO, 10) But later in the report, GAO states that since 1995 OCR has 
“implemented a more cooperative approach to their reviews” and that 
“under this approach, a letter of findings is issued only when problems 
remain unsolved” (GAO, 12) (emphasis added). 

There is a significant difference between finding a violation of the law 
(i.e. noncompliance) and finding “problems” with a school’s procedures, a 
difference that GAO fails to explain. Furthermore, OCR cannot make a 
finding of a violation finding against a school district without first issuing 
a letter of findings, a practice the agency has virtually abandoned in the past 
several years. Finally, it is highly doubtful that OCR followed its procedures 
by issuing letters of findings of violations, as GAO stated, in the five cases 
that GAO reviewed in depth. At the very least, the apparent contradictions 
in these statements should have prompted a discussion about how OCR 
procedures were applied. 

The GAO report also refers to the three relevant OCR policy documents 
issued in 1970, 1985, and 1991, correctly noting that the 1991 document 
specifically adopts a three-part test from the Castaneda court decision that 
set broad standards for determining “whether a school district has adopted 
a satisfactory method for teaching children with limited English proficien- 
cy.” The report also correctly notes “OCR did not promulgate Castanedas 
requirements as regulations, instead setting them forth in policy docu- 
ments.” But the policy documents that OCR follows were intended to pro- 
vide schools great flexibility in complying with Title VI. The GAO discus- 
sion does not raise questions about 

■ Hovi OCR’s interpretations of these policies changed to be more restric- 
tive after 1992, or 

■ Why OCR has not published its requirements as regulations. 

What was the point of Congress having GAO carry out this “investigation” 
if the results simply were to tell Congress what it already suspected? 

Furthermore, the GAO report mentions in footnote 32 that “In 
December 1999, OCR made available another resource for understanding 
tide VI requirements — Programs for English Learners: Resource Materials for 
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Planning and Self Assessments — available on the Internet” (emphasis added). 
But GAO s report provides no analysis of any information in the OCR 
guide, which spells out in chapter and verse many of the specific require- 
ments that OCR has placed upon schools since 1992. (See the policy brief: 
"Federal Control Out of Control: The Office for Civil Rights’ Hidden Poli- 
cies on Bilingual Education” for an extensive analysis of OCR application 
of these requirements in 160 school systems.)^ 

OCR and GAO point to the Lau and Castaneda decisions as providing 
support for OCR’s current practices in enforcing Title VI on this issue. But 
those decisions were based on far different facts than currently exist, and 
both decisions established broad principles. How could the courts antici- 
pate the zeal of unbridled regulators to expand sub-regulatory requirements 
ad infinitum} It is highly doubtful that any federal court would look favor- 
ably on the intrusive regulatory activities of today’s OCR. 

OCR Currendy Imposes Burdensome Requirements 

There is substantial evidence that OCR continues to impose burdensome 
requirements on schools. A recent case illustrates OCR’s efforts to require 
a school system in Southern California to agree to an onerous compliance 
agreement. In spite of numerous site visits and voluminous data requests by 
OCR, the case has remained unresolved for four years. OCR has never 
issued a letter of findings to the district, nor any other formal document 
specifying exactly where the district may have violated Tide VI. Yet OCR 
repeatedly pressured district officials to sign a "resolution agreement” that 
would, for all practical purposes, place the district into receivership to OCR 
staff. 

To their credit, school officials have not yet signed that agreement or any 
other with OCR, despite OCR pressure and the expenditure of more than 
$500,000 in legal fees. In late 2000, OCR offered the district a more con- 
densed but equally intrusive agreement. One of the most egregious require- 
ments in the condensed agreement read: 

The District will schedule periodic meetings between OCR representatives 
and individual school site resource teachers responsible for services to LEP 
students (and any other appropriate school or District staff members). At 
these meetings, the resource teachers vrill describe to OCR the criteria by 
which LEP students are placed with teachers, how LEP students are provid- 
ed English language development (ELD) in comprehension, speaking, read- 
ing and writing and how they are provided simultaneous access to grade level 
curriculum . The resource teachers will also explain how mainstreamed LEP 
students are monitored and the outcomes of that monitoring. 

OCR will meet with the school site resource teachers from three to five 
schools in the District once each school year during the 2000-01 school year 
and during the 2001-02 school year. These schools will be selected from 
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District Title I schools. At these meetings the discussion between OCR and 
the resource teachers will focus on the topics described in the preceding para- 
graph. By November 15, [2001,] OCR and the District will agree upon the 
specific schools that will be represented, and the schedule for the meetings, 
through good faith discussion. 

After each set of these meetings, OCR will provide the District with a writ- 
ten summary of observations. The District and the OCR shall meet and dis- 
cuss this item but in all respects the District shall retain its discretion to 
implement sound education programs and goals consistent with local 
resources, community needs, and Tide VI. 

It is difficult to square such OCR intrusiveness into local school concerns 
with legitimate enforcement of the civil rights laws. Should Congress 
assume that OCR staff are qualified to make judgments about whether the 
procedures schools are following in teaching LEP students are correct? 
Neither Title VI nor any court decisions provide OCR vrith such specific 
authority. GAOs failure to bring OCR to task on such matters amounts to 
a dereliction of its duty to provide a complete and honest accounting to 
Congress. 

GAO Downplayed Data from School Officials 
That Criticized OCR’s Practices 

GAO offered districts the opportunity to evaluate OCR’s activities and 
received responses to such inquiries from 245 school districts. In its analy- 
sis of these data, GAO emphasized positive responses and downplayed data 
that were critical of OCR. For example, the sidebar conclusion at the bot- 
tom of page 24 announced, in large, bold type: 

OCR Staff Generally Did Not Pressure Districts to Adopt a Bilingual 
Approach and Were Courteous and Professional 

The discussion about OCR’s interactions with school districts declared: 

Over three-fourths of the school districts responding to our survey (77 per- 
cent) reported that when investigating cases OCR staff did not appear to 
favor bilingual education over English-based instruction. 

But put another way, we learn that in 40 school systems (23 percent, or 
almost one in four cases), OCR did give the appearance of favoring bilin- 
gual education programs. That should have been a major concern for GAO, 
but it was not. Apparently, GAO felt it was generally okay for OCR to 
pressure only 40 school systems to adopt bilingual education programs. 
This is an interesting perspective given OCR’s often-declared policy that 
schools may adopt any expert-based program to meet Title VI require- 
ments. 
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GAO Structured Its Survey and Report 
to Minimize Criticisms of OCR 

The GAO report states, on page 29: 

As part of our survey, we gave school district officials the opportunity to 
make suggestions on how OCR could improve its investigation procedures 
and to offer any additional comments about OCR’s investigation of their 
school district. Of the 245 questionnaires returned by school districts, almost 
half (47 percent) contained comments on what OCR could do to be more 
effective or improve its investigative process, and over half (53 percent) made 
additional comments about OCR’s investigation of their school district. 
Although district officials generally reported positive interactions between 
their school district and OCR, some respondents commented on the type 
of problems they encountered during OCR’s investigation process. Several 
of the problems reported in the survey comments also surfaced in our case 
investigations, (emphasis added). 

This language masks the scope and degree of complaints that school dis- 
tricts recorded against OCR. On page 29, GAO assigned complaints from 
school districts to seven categories: (1) OCR “applied pressure”; (2) OCR’s 
communications were untimely or inadequate; (3) districts lacked sufficient 
resources to address problems; (4) OCR made burdensome data requests; 
(5) OCR investigators lacked educational expertise in a variety of areas; (6) 
OCR was not clear enough about case closure practices; and (7) state and 
federal requirements differed. 

What GAO did not report is that a total of 195 complaints were made 
by the responding school districts. It is also significant that school officials 
voiced so many complaints even though GAO failed to provide an appro- 
priate survey model for registering such complaints.GAO chose to use a 
general, open-ended approach that is more suitable for a technical assis- 
tance report than for a serious audit. GAO then failed to provide any mean- 
ingful followup or analysis of these problems. 

GAO Accepted Bureaucratic Responses from OCR 
to the Serious Concerns Raised by School Officials 

GAO relied exclusively on a series of non-responsive answers from OCR to 
dismiss the legitimate (and uninvestigated) concerns of school officials. For 
example, in response to districts’ concerns that “OCR applied pressure,” 
OCR stated: 

Although OCR is increasingly working in collaboration with school districts 
and reviews are now partnership oriented, it is stiU OCR’s responsibility to 
ensure that school districts comply with the law. (GAO, 30) 

Such a response is justified when there are specific violations of clearly 
enunciated legal standards. However, under the circumstances, the same 
ones that led Congress to request this investigation, OCR’s response reveals 
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its Orwellian character. 

Furthermore, when responding to a very real complaint that “Districts 
lacked sufficient resources to address problems,” OCR states: 

Serving students with limited English proficienq^ takes time and costs 
money. OCR attempts to be flexible with school districts. For example, in the 
negotiated agreements, OCR gives school districts time to hire necessary 
qualified teachers. In some cases, OCR has worked with universities to put 
teacher-training programs into place; it has also worked to increase certifica- 
tion opportunities for teachers. (GAO, 30) 

This dismissive statement exemplifies the total disregard OCR shows for 
real life, that is, for facts that do not fit neatly into its computer-pro- 
grammed corrective action plans. 

OCR provided similar non-responsive answers to the other school offi- 
cials’ concerns, but perhaps the most insulting response to educators was to 
the complaint that “OCR investigators lacked educational expertise in a 
variety of areas.” To that very legitimate criticism, OCR responded: 

OCR is addressing this issue through conferences for OCR enforcement 
staff. OCR has established employee groups organized by subject matter to 
discuss policy and legal decisions related to students with limited English 
proficiency. Through these groups, guest speakers and other resources are 
now readily available. In addition, the Lau Articulation Project produced a 
list of educational resources that OCR enforcement staff use. (GAO, 30) 

It is remarkable that OCR’s responses in this and other areas failed to 
raise the investigative curiosity of the GAO auditors. For example, GAO 
does not mention or did not bother to find out that OCR investigators 
come from a broad scope of educational backgrounds. A few are former 
educators; some have advanced degrees and other professional credentials. 
But a significant number of OCR investigators lack a college degree, and 
some do not even have a high school education. Nor did GAO show any 
curiosity about the backgrounds of people allegedly providing training to 
OCR staff on educational programs for LEP students. It is common 
knowledge in OCR that its staff seek and receive advice on these matters 
almost exclusively from advocates for bilingual education programs. 

It is difficult to choose which of OCR’s responses was the most disin- 
genuous, but the agency’s answer to schools’ concerns that “OCR made 
burdensome data requests” is a likely contender: 

OCR is refining its approach to data requests. Having moved to the Case 
Resolution Manual, OCR’s emphasis is now on resolving compliance issues 
in partnership with school districts instead of on making findings. This often 
results in less burdensome data requests. (GAO, 30) 

OCR adopted the Case Resolution Manual in early 1994. How can 
OCR credibly claim that a 1994 procedural change is just now being used 
to resolve complaints about overly burdensome data requests that have aris- 
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en in the year 2000? How can GAO accept such a meaningless response? 
The GAO report does admit that 22 school districts complained about 
OCR’s burdensome data requests and “One [school] official reported that 
it took over 600 staff-hours to collect the data requested by OCR,” but pro- 
vides no analysis about the severity of these complaints. (GAO, 29) 

GAO gave similar superficial treatment to the remaining areas in which 
school officials expressed concerns: “OCR’s communications were untime- 
ly or inadequate” and “OCR was not clear enough about case closure prac- 
tices.” In fact, there is no indication in the GAO report that GAO investi- 
gators followed up with any of the school officials who expressed serious 
concerns about OCR’s procedures. 

GAO’s Conclusions About OCR’s Practices 
Are Superficial and Biased 

In its Concluding Observations section, the GAO report placed the final 
coat of whitewash on its “investigation” of OCR’s activities: 

[Gjuidance from OCR provides the framework and standards that school 
officials must meet to ensure that students with limited English proficiency 
have a meaningful opportunity to participate in public education. School dis- 
tricts have the flexibility to select methods of instruction that they deem 
will produce the best results for their students, so long as th^ meet OCR 
requirements (emphasis added). (GAO, 31) 

The congressional request for the GAO investigation was triggered in 
part because of serious questions about the standards OCR uses and the 
requirements the agency imposes. The GAO report does not attempt to 
clarify those standards and ignores or excuses OCR’s burdensome “correc- 
tive” requirements. GAO carelessly assumes that these sub- regulatory 
requirements “ensure that students with limited English proficiency have a 
meaningful opportunity to participate in public education.” 

Finally, GAO concludes: “There have been some problems, however, 
with OCR’s working relationships with districts, which OCR acknowl- 
edges and is taking steps to improve.” (GAO, 31) 

This single reference to OCR’s “problems” glosses over the civil rights 
agency’s high-handed interpretation of the laws. These law^ should never 
have been construed to allow OCR such extraordinary powers over local 
school programs, giving investigative staff a free hand to judge and direct 
local school decisions. Does GAO buy into the notion that to criticize 
OCR’s practices is to be against civil rights enforcement? This wrong and 
shortsighted view deprives Congress of essential information as that leg- 
islative body considers enacting reforms. It may also result in trapping yet 
another generation of LEP students in failed programs. 
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